AI Data Vision

OpenAI and Azure OpenAI are leading the generative AI market, gaining significant traction. OpenAI’s groundbreaking GPT models and continuous advancements make it a frontrunner in language generation. Azure OpenAI, a collaboration between OpenAI and Microsoft Azure, brings generative AI to a broader businesses audience through scalable cloud infrastructure. Together, they drive adoption, shape the future of generative AI, and fuel transformative applications. Today, we will discuss the updates of Azure OpenAI (AOAI) for June 2023.

OpenAI with your own data

The addition of the “Azure OpenAI on your Data” functionality is the main enhancement! It involves the integration of Enterprise Search, enabling companies to use LLMs on their own data while ensuring compliance with internal data policies. This allows business units to harness the power of these models to obtain justified answers (with citations) to their questions based on company documents. Direct use cases can be envisioned for support functions such as legal, sales, and human resources. For instance, being able to query contracts, compare them, search for specific clauses, and so on.

In the Chat interface, you can now add your own data (see images below). Adding your data involves creating a Cognitive Search index from a container with your files on a Storage account (The supported file formats for local index creation are: .txt, .md, .html, .pdf, .docx, and .pptx : Microsoft Documentation). This index will be used to retrieve relevant information based on the user’s question.

Once the index is set up, it is possible to deploy a WebApp (see below). In my example, I created an index using the first novel of the Harry Potter saga (in French). I have attached some screenshots showcasing examples of question-answer pairs.

Here is the video “Add Your Own Data” from Microsoft:

Plugins

The second most anticipated feature is the ability to add plugins to Azure OpenAI. Plugins are extensions designed to expand the capabilities of the OpenAI chatbot. They allow the AI to access up-to-date information or utilize third-party services. As of now, this feature is not yet available in the Studio but is expected to be introduced in July (limited preview).

Microsoft explains that while the July preview will focus on pre-built plugins (such as Bing Search, Azure Cognitive Search, Azure SQL, Azure Cosmos DB, and Microsoft Translator), customers will be able to use their own plugins in the future! If you want to learn more about how to create plugins for OpenAI (applicable to Azure OpenAI according to Microsoft), I suggest you check out this tutorial.

Here is the Microsoft presentation:

Quotas

Another addition is the implementation of quotas. The purpose of quotas is to limit abuse and ensure proper usage by customers. Quotas can be observed at the subscription and region level, and their monitoring can be done across different deployed models. Quotas need to be specified when deploying models and allow you to customize them according to your needs.

It is also possible to request an increase in Azure OpenAI TPM (tokens-per-minute) quotas through the following form: Request for Quota Increase. Note that at the time of writing this article, your request may take some time to process as Microsoft is experiencing high demand.

Content filtering

Last but not least, the addition of content filters allows clients to specify a list of prohibited (or authorized) terms and expressions. These filters ensure that the content is appropriate for its intended audience. The filters are customizable by the client, although I personally find the options of Low, Medium, and High a bit ambiguous. I can only recommend trying them out for yourself after reading Microsoft documentation.

Please note that by default, client does not have access to all filter options. The request can be made here: Request to modify Content Filters.