AI Data Vision

OpenAI in Azure

If you’re interested in AI, you’ve undoubtedly heard of OpenAI and its AI models (ChatGPT, DALL-E, Codex). The most curious among you also know that OpenAI is in partnership with Microsoft for the use of the Azure cloud and AI development. The goal here is to accelerate research around generative AI and make it accessible to everyone.

Recently, many announcements have been made about OpenAI and its integration into Microsoft services, whether in Azure, alongside Cognitive Services (Azure OpenAI), through the Bing search engine (New Bing), or through the Modern Workplace (Microsoft 365 Copilot).

This post will cover Azure OpenAI, including the advantages of using Azure cloud for OpenAI, a presentation of Azure OpenAI Studio, and a few examples demonstrating how Azure OpenAI can be utilized with other cognitive services.

Deploy Azure OpenAI

To deploy Azure OpenAI, you currently need to have:

• An Azure subscription.
• To fill out a request form (to gain access to AzureOpenAI). Once the access is granted, you can create the resource.

You can use Azure OpenAI Studio to experiment and manage your models as well as train custom models. The resource also provides a key and endpoint that allows you to make API calls from your application.

Azure benefits

Microsoft has reinforced its PaaS strategy around AI by integrating OpenAI into its cloud. This integration offers various benefits :

• Better user experience through Azure OpenAI Studio.
• Interfaces for fine-tuning and monitoring.
• Custom models traning based on user data, which will not be used or communicated for training other models according to Microsoft.

Additionally, Microsoft demonstrates responsible AI engagement through content filtering, which is modifiable by request, and transparency notes.

The integration of OpenAI in Azure offers security measures such as role-based access control (RBAC), virtual networks, and a choice in Azure regions for data governance.

Azure OpenAI Studio

Azure OpenAI Studio presents a Playground tab for the users to test their use cases. This playground is divided into two tabs:

1. Completions offers preloaded examples including summarization, content generation, text classification, sentiment analysis, language translation, including code.
2. Chat allows the user to test ChatGPT. It is possible to provide the conversational AI with a “system message” as well as exemples to constrain its responses.

Important: at the time, Chat features is only availables in 2 Azure regions (EastUS & SouthCentralUS). It will become accessible from other regions in the future.

This playground also shares a “Parameters” tab (framed in yellow on image#1) for fine-tuning the model and includes key parameters such as:

• Temperature: varies from 0 to 1 and controls randomness. A lower value indicates the model will produce more deterministic responses. An higher value increases the responses creativity.
• Max length (tokens): indicates the max number of tokens per response. This length is shared between the prompt and the model’s response. A token is equivalent to 3-4 characters for an english text, this definition is very important for the pricing considerations (Pricing Calculator + Azure OpenAI pricing).
• Stop sequences: makes responses stop at a desired point like the end of a sentence. It call also be a “;” to indicate the end of a SQL query.
• Top-p: varies from 0 to 1 and controls randomness by selecting likelier tokens. Increasing Top-p will allow the model to choose token with low likelihood.

Here are some screenshots to present the “Completions” interface through the examples provided by Microsoft or by my experiments.

Here is the “Chat” interface with an example of an AI assistant taking on the role of an advisor for a french telecom company:

Azure OpenAI Studio also shares a Management tab to control the deployed models and manage files in case the user wants to train a customized model.

First, the Deployment tab allows user to see the deployed models.

The Models tab offers the option to deploy pretrained models or train a custom model. Text models for GPT and code models for Codex are available.

Quick explanation on How to choose the model ?

• Models A through D increase in complexity and provide higher quality responses, but come at a higher cost and inference time (refer to diagram below). Davinci generally generates better responses, but it’s recommended to try out each model to determine which best fits your needs and budget.
• The same concept applies to Codex models, with Cushman being more suitable for real-time applications due to its speed.

Finally, the File Management tab allows user to upload data to train and validate the custom models.

Azure OpenAI Integration

A major benefit of integrating OpenAI into Azure is the ease of combining OpenAI with other cognitive services to address business challenges.

Example 1 – Call Center: Azure OpenAI can be used in conjunction with Azure Speech to analyze conversations between the advisor and the client. Azure Speech-to-text transcribes the audio into text that can be analyzed by Azure OpenAI. The model can then be used to classify the conversation, perform sentiment analysis, generate real-time questions to suggest to the advisor, and extract information (summary generation, analysis) to feed into a customer database.

Example 2 – Enterprise GPT: Azure OpenAI is used in conjunction with Azure Cognitive Search to directly query internal company documents (HR, legal, marketing, etc.). Azure Cognitive Search indexes documents pre-defined by the company (with the help of a Form Recognizer for OCR if necessary) or even entire databases. Once the documents are indexed, Azure OpenAI can optimize the user query to query the index and generate a response while citing the supporting documents for the response (see just below; Github repo).

In conclusion, Azure OpenAI provides businesses with powerful AI capabilities that can be easily integrated with other cognitive services to address various business challenges. Its ability to analyze conversations, classify information, perform sentiment analysis, and extract information from documents can greatly enhance customer experiences and improve operational efficiency. By leveraging Azure OpenAI, businesses can unlock new possibilities and gain a competitive advantage in their respective industries.