AI Document Assistant

The AI Document Assistant allows users to connect to their Microsoft Foundry account and utilize the LLM models they are subscribed to in the Therefore™ Web Client and Therefore™ Viewer.

More information about the feature as displayed in the Therefore™ Viewer can be found on the following page:

AI Document Assistant Pane

The feature has a trial mode that allows users to utilize a fixed amount of tokens for testing. For production, administrators need to add existing Microsoft Foundry credentials.

Right-clicking on the AI Document Assistant node under Integrations in the Therefore™ Solution Designer opens a menu with the options Settings, Consumption Log, and Security.

Settings

Opens the AI Document Assistant dialog containing the Configuration and Prompts tab. Double-clicking the AI Document Assistant node opens same dialog.

Configuration

In this tab, basic settings such as Microsoft Foundry credentials and instructions for the Therefore™ AI Document Assistant are saved.

Enable AI Document Assistant (Checkbox)

Enable use of the AI Document Assistant feature for the Therefore™ tenant.

Category selection (List and browse button)

In addition to activating the feature, categories need to be added manually in order for end users to be able to use the AI Document Assistant. Added categories are listed in this dialog. To add more categories, click the browse button to open a picker and select them from the tree view.

Trial mode

If Trial mode is selected, a number of free tokens is available for users to try the AI Document Assistant feature. The number of available tokens scales with the number of Named and Concurrent licenses.

Note: Consumption

The total number of tokens assigned to a Therefore™ tenant is available for all users with access to the AI Document Assistant. This means it is possible for a single user to consume every token.

Once all trial tokens are used trial mode is ended. Available tokens are displayed in the AI Document Assistant configuration dialog, and token consumption can be monitored by checking the consumption log.

Note: Tokens

In English, one token is approximately equal to 4 characters.
The supported context window supported by the model in trial mode is up to 1,050,000 tokens. The context window describes all referenced files, user prompts, and responses that can be considered by the model in the current conversation.
Maximum model output of the model in trial mode is up to 128, 000 tokens. On production, different models have different maximum output. Context windows also differ by model.

The trial can be interrupted and started again but the trial token consumption is not reset.

Microsoft Foundry Account

For production use of the AI Document Assistant feature, existing Microsoft Foundry Credentials have to be entered.

Credential

Select a web credential from the dropdown list. If no credential for authentication against Microsoft Foundry exists yet, select the option 'New Web Credential' at the bottom of the list of credentials.

Credentials are stored under the Credentials node in the Therefore™ Solution Designer.

Credentials

The credential needs to be configured as follows:

Credential ID

Give the credential an intuitive name.

Base URL

This is part of the project endpoint URL that has to be copied from the Microsoft Foundry project. For example, if the following project URL is used the part marked in yellow is to be entered (https://moyaware-foundry.ai.azure.com):

https://moyaware-foundry.ai.azure.com/api/project/my-project

Type

The type needs to be set to Bearer Token.

Bearer token

Copy the API key from Microsoft Foundry into this field.

Foundry Project

Enter the last part of the endpoint URL into this field.

If the following endpoint URL is used the part marked in yellow (my-project) is to be used:

https://moyaware-foundry.ai.azure.com/api/project/my-project

Model

Select an LLM model that should be used in the AI Document Assistant. Any model that supports the Response API can be used. Models need to be built and deployed in Microsoft Foundry before they show up in the dropdown menu.

Instructions

Either use the predefined instructions for the AI Document Assistant or enter customized instructions for the AI Document Assistant to follow. The default instructions use Task Instruction Formatting but it is not required.

Prompts

In this tab, administrators can define prompts the users can select when using the AI Document Assistant, or view defined prompts.

Add Filter (button)

Add a filter to filter the defined prompts by category. The available options are listed below.

No Filter

View all prompts.

All categories

View prompts that are available for all categories.

No category

View prompts that have not been assigned to any categories.

Select category

Select categories from a picker to filter prompts assigned to these categories.

Search (bar)

Use the free text search to search for search terms contained in prompt titles and bodies.

	Add Prompt (button) Click this button to add a new prompt to the list.
	Edit Prompt (button) If a prompt is selected this button becomes available. Click it to edit the selected prompt.
	Delete Prompt (button) Click this button to delete a selected prompt.
	Move Up (button) Move a selected prompt towards the top of the list.
	Move Down (button) Move a selected prompt towards the bottom of the list.

Prompts and Categories (table)

View the prompt titles of the defined prompts in the left table column and the category or categories they are assigned to in the right table column.

Categories can be added by clicking the Browse button next to the prompt and selecting the category from the tree view picker that opens. If a filter is active, a filter icon appears on the right side of the table and the Browse buttons are unavailable.

The AI Document Assistant comes with several predefined prompts that can be used out of the box or customized. A title and description of the predefined prompts are listed below.

Prompt title	Description
Short summary	Instructs the LLM to provide a summary of the document in 3-4 sentences.
Detailed summary	Generates a more detailed summary of the document including an overview in 3-5 sentences.
Identify actors	Sends instruction to list all people and organizations mentioned in the document.
Compare Documents	Compares provided reference files and lists similarities and differences.
Dates and deadlines	Sends instructions to extract relevant dates and timeframes from the document.
Action Items	Identifies relevant actionable items such as tasks in the document

Prompt preview

Review and edit the prompt body.

Translation

Opens a translation dialog in which administrators can add translations for prompt titles and prompt bodies.

Translation

Consumption Log

Selecting this option opens the AI Document Assistant Consumption Log dialog. The following options can be configured:

From (calendar picker)

Set the start date for the consumption log by selecting it in the calendar.

To (calendar picker)

Set the end date for the consumption log by selecting it in the calendar.

User selection (dropdown menu)

Select a user by their username from the dropdown menu to create a log of their token consumption. By default, <All Users> is selected. If this option is selected, a log of the token consumption of all users is generated.

Copy report to clipboard (button)

Copy the current view to clipboard.

Export report data (button)

Export the current view as a CSV file.

Redraw report (button)

Create a new report after changing the settings.

Consumption log (table)

The first four columns displayed in the log contain data from Therefore™.

Depending on the AI model used, more columns are displayed. These columns are populated from responses. For these columns, system names are shown.

Date

Displays the date/time at which token consumption occurred.

Username

Displays the username of the Therefore™ user who consumed the tokens.

Document Number

Displays the document number of the Therefore™ document that serves as the main reference of the conversation the token was used in.

Trial Tokens

If the token consumption occurred in trial mode, the number of consumed trial tokens are displayed in this column.

Security

View and edit role-based access for anything below this node.

To use the AI Document Assistant as an end user, users will need at least the predefined Reader role or a role with equivalent permissions on the AI Document Assistant node.

Security

Advanced Settings

The following Advanced Settings can be configured for the AI Document Assistant.

AI Output Token Limit

This setting limits the maximum number of output tokens for the AI Document Assistant. The default setting is 5000 tokens.

Character Limit per Prompt

This setting limits the maximum number of characters in user prompts. The default setting is 10 000 characters.

Character Limit per Reference

This setting limits the maximum number of characters per each reference the user can add. The default setting is 100 000.