Creating a custom Language Model

    The Language Model is used to help disambiguate between probable words and phrases. For example, did you say "two" or "to" or "too"?

    You can use the API to train a custom Language Model which will include any custom vocabulary, like product names and person names, unique to your application. You can also boost accuracy for common phrases and speaking patterns. For example, speaking patterns in a conversation versus a lecture/presentation are very different, and a Language Model can capture this difference.

    Preparing your Training Data

    To train a Language Model, you need to prepare a bunch of text data. The text data can either be a list of custom vocabulary terms like product names, common phrases that you want to boost accuracy for, or (best case) historical transcriptions that you may have which contain the custom vocab/common phrases you want to boost accuracy for -- this is the best case scenario because it helps the Language Model learn the context in which these words/phrases appear.

    Making the API Request

    You can start training a Language Model by making the following API request to the /v2/lang-model endpoint. The "text" param should be a list of words or phrases, with each word or phrase as an individual string in the array. Punctuation and other text formatting is ok.

    curl --request POST \
      --url https://api.assemblyai.com/v2/lang-model \
      --header 'authorization: YOUR-API-TOKEN' \
      --header 'content-type: application/json' \
      --data '
        {
            "text": ["I want to cancel my account.", "Welcome to Verizon how can I help you.", "CT Scan"],
            "name": "foo_bar_lm"
        }'

    You'll get a response like this:

    {
        "closed_domain": false,
        "id": "145557-e909-491c-bc03-46d619292156",
        "name": "foo_bar_lm",
        "status": "queued",
        "webhook_status_code": null,
        "webhook_url": null
    }

    Checking on the Status of your Language Model

    As your Language Model trains, it's "status" will go from "queued" to "training" to "trained". If an error is encountered while training your Language Model, it'll go to a "status" of "error".

    You can poll for the status of your Language Model using the below API request, or you can include a "webhook_url" param in your POST which will trigger a webhook to be sent to the webhook URL you supply whenever your Language Model goes to a "status" of "trained" or "error". The webhook will be a POST to your URL with the following JSON body: {"model_id": "your-model-id", "status": "trained"}.
    curl --request GET \
      --url https://api.assemblyai.com/v2/lang-model/145557-e909-491c-bc03-46d619292156 \
      --header 'authorization: YOUR-API-TOKEN' \
      --header 'content-type: application/json'

    Once your "status" is shown as "trained", you can begin to use your Language Model for transcription.