Knowledge bases

What is a knowledge base?

A knowledge base is a collection of data sources. Knowledge bases are used to empower your Motifs with contextual knowledge of the data that may not be part of the model’s original training data. Each knowledge base may consists of multiple data sources. Each data source can be configured to synchronize on a set schedule. This ensures that a Motif’s contextual data is kept up to date with the changing data.

Motific enables you to create KB configurations to connect with your data sources so that the model’s response gets relevant contextual information from the data sources.

The knowledge base data sources can be one of the following:

  • SharePoint - SharePoint-Online sites where you have different files and folders that can be the data source for the knowledge base.
  • Public websites - Public websites are the websites available on the general internet. These websites can be added as data sources to a KB.

On KB page you can:

Create a new knowledge base

  1. To navigate to the knowledge base connections page, go to Connections menu » Knowledge bases tab. add KB

  2. To create a new knowledge base (KB) connection, click on the Create new knowledge base button. A page with fields to create new KB opens. add KB

  3. Next, add data sources for your KB. You have the option of adding one or multiple of the following two data source types:

Add SharePoint

Motific supports SharePoint-Online to be added a data source. Also, Motific Azure AD App-Only authentication. Procedure to grant access via Azure AD App-only is available in the Microsoft documentation

  • To add a SharePoint as a data source, click Add SharePoint option.

  • Provide the URL for the SharePoint site that you want to add.

  • Enter the source path- the path to the folder or document library you want to index.

  • Provide a Data source name for your SharePoint data source.

    add KB

  • Enter the Azure app ID for the site.

  • Enter the Azure tenant ID for the site.

  • Enter the Certificate thumbprint for the SharePoint site.

  • Provide Certificate PEM (Privacy Enhanced Mail). Drag and drop or click on the drop area to upload the PEM certificate.

    add KB

  • Define a schedule of how often the data source should be updated by selecting an option from the drop-down. You have the following options:

    • One-time - The data source sync begins as soon the KB is created and is a one-time sync.
    • Weekly- Specify the day of the week and time of the day when the data source should be updated with any new content.
    • Monthly- Specify day of the month and time of the day when the data source should be updated with any new content.
    • Daily- Specify the time of the day when the data source should be updated with any new content.
  • Verify all the details, then to add the data source , click the Add SharePoint button.

Add public website

Public websites are the websites available on the general Internet. These websites can be added as data sources to a KB.

  • To add a website as a data source, click Add Website button. add ws

  • Provide the source URL for the website that you want to add.

  • Provide a name and description for your website.

  • Define a schedule of how often the data source should be updated by selecting an option from the drop-down. You have the following options:

    • One-time - The data source sync begins as soon the KB is created and is a one-time sync.
    • Weekly- Specify the day of the week and time of the day when the data source should be updated with any new content.
    • Monthly- Specify day of the month and time of the day when the data source should be updated with any new content.
    • Daily- Specify the time of the day when the data source should be updated with any new content.
  • Verify all the details, then to add the data source, click the Add public website source button.

  • When you are done adding the data source(s), you can view the details of the data source added in the knowledge base summary section. You can also delete the data sources in the summary section by clicking the red delete icon.

  • Click the Create Knowledge base button to add the data sources to the knowledge base.

View KB details

When you navigate to the Knowledge base menu the existing KBs are displayed.

You can view the KB name, number of data sources added to the KB when it was created. The data source names can be viewed by hovering over the data sources. Also, created by and the last modified time.

To view each KB detail, follow the steps below:

  1. To view KB details, click on the KB name.

  2. The page with the KB metrics opens.

  3. Here, you can view the following:

    • Knowledge base details: The sync status depicts what is the status of the ingestion of the KB. You can also view the KB name, KB creation date, last updated date. The sync status can be in one of the following states- Syncing, Complete, N/A, and None.

    • Data source details: In the data source details following items can be viewed:

      • name of the data source and the link to the URL can be viewed.
      • the data source’s last updated date
      • number of records- shows the number of files processed, passed, or failed during the ingestion of the data source.

    If you see any record that has failed ingestion, then you can view the syncs. Click the View syncs button.

  4. In the data source details if you see any failure of ingestion of a data source, then you can click the Resync data source button to resync the data source.

Resync data sources

The knowledge base that you created can be viewed on the Knowledge base page. The existing KB (if any) are in the list view.

  1. To view the sync status of each data source in a KB, click on the KB whose data source sync status you want to check.

  2. In the Data source details section, when some of the records have failed ingestion, you can resync the data source.

  3. To resync the data sources, click the Resync data source button. The resync starts and the details of the sync can be seen by clicking View syncs.

View syncs

In the data source details section, to see the sync details of a data source, then follow the steps below:

  1. To view the sync details of a particular data source, click the respective data source’s View syncs button.

  2. The data source syncs section is displayed. Here, you can see all the resyncs that you have performed.

  3. The following data is displayed for each sync:

    • Sync status- The status of the sync started.
    • Sync ID- Unique ID for each re-sync request
    • Started at- The time period when the sync was started
    • Sync duration- The duration taken to ingest and sync the data source.
    • Expected- The number of expected files that need to be ingested for the data source.
    • Processed- The number of files that are already processed and ingested.
    • Failed- The number of files that have failed the ingestion process.
    • View logs- View the logs for the particular sync run.
    • View details- View the details of each and every sync that was started.

View logs

To view the logs of each sync and to find out of what kind or alert level or message is recorded, follow the steps below.

  1. Navigate to the knowledge base details of the KB.

  2. To view the sync details of a particular data source, click the respective data source’s View syncs button.

  3. To view the logs of a data source during a particular sync, click the corresponding sync’s View logs link. A small modal with the log opens.

  4. The following details can be viewed in the logs:

    • Alert level: Alert level shows the level of the alert in the log details.
      • Info: Info alert includes the information about the operations of the data source sync.
      • Error: Error alert is assigned to event logs that contain a data source sync error message.
    • Message: Here, the message of the log is displayed depending on the alert level.
    • Origin: Origin signifies what is the origin of the particular log.
    • Step: Step signifies at which step of the sync process the event was logged.
    • Reported at: The time when the alert or log was reported.

View sync details

To view the details of each sync and the status of the records/files within the data source, follow the steps below:

  1. Navigate to the knowledge base details of the KB.

  2. To view the sync details of a particular data source, click the respective data source’s View syncs button.

  3. To view the details of each record within the data source during a particular sync, click the corresponding sync’s View details link.

  4. The page with the details of each record within the data source for that sync cycle opens.

    If there are errors while syncing the data source then the errors would be diaplyes ad shown below:

  5. The list of latest ingestion with summary and list of documents opens.

  6. The summary section shows the number of documents that are in the following state:

    • Ingestion- The total number of records to be ingested.
    • Processing- The total number of records that are being processed.
    • Indexing- The total number of records that are being indexed.
    • Completed- The total number of records that have completed ingestion and are now in completed state.
    • Error- The total number of records that were errored out during ingestion.
  7. In the documents section, the following details are indicated about each document/record present in the data source.

    • Path- The path of the document.
    • Ingestion status- The ingestion status of the file. If there is a green check mark that means the ingestion is completed, otherwise it is still in progress.
    • Processing status- The processing status of the file. If there is a green check mark that means the processing is completed, otherwise it is still in progress.
    • Indexing status- The indexing status of the file. If there is a green check mark that means the indexing is completed, otherwise it is still in progress.
    • Error- This data shows if the document/record failed one of the above steps and was not synced with Motific.