---
title: Azure AI Services Toolkit
---

이 toolkit은 `Azure AI Services API`와 상호작용하여 일부 multimodal 기능을 구현하는 데 사용됩니다.

현재 이 toolkit에는 다섯 가지 tool이 포함되어 있습니다:

- **AzureAiServicesImageAnalysisTool**: 이미지에서 캡션, 객체, 태그 및 텍스트를 추출하는 데 사용됩니다.
- **AzureAiServicesDocumentIntelligenceTool**: 문서에서 텍스트, 표 및 키-값 쌍을 추출하는 데 사용됩니다.
- **AzureAiServicesSpeechToTextTool**: 음성을 텍스트로 변환하는 데 사용됩니다.
- **AzureAiServicesTextToSpeechTool**: 텍스트를 음성으로 합성하는 데 사용됩니다.
- **AzureAiServicesTextAnalyticsForHealthTool**: 의료 entity를 추출하는 데 사용됩니다.

먼저 Azure 계정을 설정하고 AI Services 리소스를 생성해야 합니다. 리소스를 생성하려면 [여기](https://learn.microsoft.com/en-us/azure/ai-services/multi-service-resource)의 지침을 따르세요.

그런 다음 리소스의 endpoint, key 및 region을 가져와서 환경 변수로 설정해야 합니다. 리소스의 "Keys and Endpoint" 페이지에서 찾을 수 있습니다.

```python
pip install -qU  azure-ai-formrecognizer > /dev/null
pip install -qU  azure-cognitiveservices-speech > /dev/null
pip install -qU  azure-ai-textanalytics > /dev/null
pip install -qU  azure-ai-vision-imageanalysis > /dev/null
pip install -qU langchain-community

import os

os.environ["OPENAI_API_KEY"] = "sk-"
os.environ["AZURE_AI_SERVICES_KEY"] = ""
os.environ["AZURE_AI_SERVICES_ENDPOINT"] = ""
os.environ["AZURE_AI_SERVICES_REGION"] = ""

​
Toolkit 생성하기

from langchain_community.agent_toolkits import AzureAiServicesToolkit

toolkit = AzureAiServicesToolkit()

[tool.name for tool in toolkit.get_tools()]

['azure_ai_services_document_intelligence',
 'azure_ai_services_image_analysis',
 'azure_ai_services_speech_to_text',
 'azure_ai_services_text_to_speech',
 'azure_ai_services_text_analytics_for_health']

​
Agent 내에서 사용하기

from langchain_classic import hub
from langchain.agents import AgentExecutor, create_structured_chat_agent
from langchain_openai import OpenAI

llm = OpenAI(temperature=0)
tools = toolkit.get_tools()
prompt = hub.pull("hwchase17/structured-chat-agent")
agent = create_structured_chat_agent(llm, tools, prompt)

agent_executor = AgentExecutor(
    agent=agent, tools=tools, verbose=True, handle_parsing_errors=True
)

agent_executor.invoke(
    {
        "input": "What can I make with these ingredients? "
        + "https://images.openai.com/blob/9ad5a2ab-041f-475f-ad6a-b51899c50182/ingredients.png"
    }
)

> Entering new AgentExecutor chain...

Thought: I need to use the azure_ai_services_image_analysis tool to analyze the image of the ingredients.
Action:
\`\`\`
{
  "action": "azure_ai_services_image_analysis",
  "action_input": "https://images.openai.com/blob/9ad5a2ab-041f-475f-ad6a-b51899c50182/ingredients.png"
}
\`\`\`
Caption: a group of eggs and flour in bowls
Objects: Egg, Egg, Food
Tags: dairy, ingredient, indoor, thickening agent, food, mixing bowl, powder, flour, egg, bowl
Action:
\`\`\`
{
  "action": "Final Answer",
  "action_input": "You can make a cake or other baked goods with these ingredients."
}
\`\`\`



> Finished chain.

{'input': 'What can I make with these ingredients? https://images.openai.com/blob/9ad5a2ab-041f-475f-ad6a-b51899c50182/ingredients.png',
 'output': 'You can make a cake or other baked goods with these ingredients.'}

tts_result = agent_executor.invoke({"input": "Tell me a joke and read it out for me."})
audio_file = tts_result.get("output")

> Entering new AgentExecutor chain...

Thought: I can use the Azure AI Services Text to Speech API to convert text to speech.
Action:
\`\`\`
{
  "action": "azure_ai_services_text_to_speech",
  "action_input": "Why don't scientists trust atoms? Because they make up everything."
}
\`\`\`
/tmp/tmpe48vamz0.wav


> Finished chain.

from IPython import display

audio = display.Audio(data=audio_file, autoplay=True, rate=22050)
display.display(audio)

sample_input = """
The patient is a 54-year-old gentleman with a history of progressive angina over the past several months.
The patient had a cardiac catheterization in July of this year revealing total occlusion of the RCA and 50% left main disease ,
with a strong family history of coronary artery disease with a brother dying at the age of 52 from a myocardial infarction and
another brother who is status post coronary artery bypass grafting. The patient had a stress echocardiogram done on July , 2001 ,
which showed no wall motion abnormalities , but this was a difficult study due to body habitus. The patient went for six minutes with
minimal ST depressions in the anterior lateral leads , thought due to fatigue and wrist pain , his anginal equivalent. Due to the patient's
increased symptoms and family history and history left main disease with total occasional of his RCA was referred for revascularization with open heart surgery.

List all the diagnoses.
"""

agent_executor.invoke({"input": sample_input})

> Entering new AgentExecutor chain...

Thought: The patient has a history of progressive angina, a strong family history of coronary artery disease, and a previous cardiac catheterization revealing total occlusion of the RCA and 50% left main disease.
Action:
\`\`\`
{
  "action": "azure_ai_services_text_analytics_for_health",
  "action_input": "The patient is a 54-year-old gentleman with a history of progressive angina over the past several months. The patient had a cardiac catheterization in July of this year revealing total occlusion of the RCA and 50% left main disease, with a strong family history of coronary artery disease with a brother dying at the age of 52 from a myocardial infarction and another brother who is status post coronary artery bypass grafting. The patient had a stress echocardiogram done on July, 2001, which showed no wall motion abnormalities, but this was a difficult study due to body habitus. The patient went for six minutes with minimal ST depressions in the anterior lateral leads, thought due to fatigue and wrist pain, his anginal equivalent. Due to the patient's increased symptoms and family history and history left main disease with total occasional of his RCA was referred for revascularization with open heart surgery."
The text contains the following healthcare entities: 54-year-old is a healthcare entity of type Age, gentleman is a healthcare entity of type Gender, progressive angina is a healthcare entity of type Diagnosis, past several months is a healthcare entity of type Time, cardiac catheterization is a healthcare entity of type ExaminationName, July of this year is a healthcare entity of type Time, total is a healthcare entity of type ConditionQualifier, occlusion is a healthcare entity of type SymptomOrSign, RCA is a healthcare entity of type BodyStructure, 50 is a healthcare entity of type MeasurementValue, % is a healthcare entity of type MeasurementUnit, left main disease is a healthcare entity of type Diagnosis, family is a healthcare entity of type FamilyRelation, coronary artery disease is a healthcare entity of type Diagnosis, brother is a healthcare entity of type FamilyRelation, dying is a healthcare entity of type Diagnosis, 52 is a healthcare entity of type Age, myocardial infarction is a healthcare entity of type Diagnosis, brother is a healthcare entity of type FamilyRelation, coronary artery bypass grafting is a healthcare entity of type TreatmentName, stress echocardiogram is a healthcare entity of type ExaminationName, July, 2001 is a healthcare entity of type Time, wall motion abnormalities is a healthcare entity of type SymptomOrSign, body habitus is a healthcare entity of type SymptomOrSign, six minutes is a healthcare entity of type Time, minimal is a healthcare entity of type ConditionQualifier, ST depressions in the anterior lateral leads is a healthcare entity of type SymptomOrSign, fatigue is a healthcare entity of type SymptomOrSign, wrist pain is a healthcare entity of type SymptomOrSign, anginal is a healthcare entity of type SymptomOrSign, increased is a healthcare entity of type Course, symptoms is a healthcare entity of type SymptomOrSign, family is a healthcare entity of type FamilyRelation, left main disease is a healthcare entity of type Diagnosis, occasional is a healthcare entity of type Course, RCA is a healthcare entity of type BodyStructure, revascularization is a healthcare entity of type TreatmentName, open heart surgery is a healthcare entity of type TreatmentName
Action:
\`\`\`
{
  "action": "Final Answer",
  "action_input": "The patient's diagnoses include progressive angina, total occlusion of the RCA, 50% left main disease, coronary artery disease, myocardial infarction, and a family history of coronary artery disease."
}



> Finished chain.

{'input': "\nThe patient is a 54-year-old gentleman with a history of progressive angina over the past several months.\nThe patient had a cardiac catheterization in July of this year revealing total occlusion of the RCA and 50% left main disease ,\nwith a strong family history of coronary artery disease with a brother dying at the age of 52 from a myocardial infarction and\nanother brother who is status post coronary artery bypass grafting. The patient had a stress echocardiogram done on July , 2001 ,\nwhich showed no wall motion abnormalities , but this was a difficult study due to body habitus. The patient went for six minutes with\nminimal ST depressions in the anterior lateral leads , thought due to fatigue and wrist pain , his anginal equivalent. Due to the patient's\nincreased symptoms and family history and history left main disease with total occasional of his RCA was referred for revascularization with open heart surgery.\n\nList all the diagnoses.\n",
 'output': "The patient's diagnoses include progressive angina, total occlusion of the RCA, 50% left main disease, coronary artery disease, myocardial infarction, and a family history of coronary artery disease."}


---

<Callout icon="pen-to-square" iconType="regular">
    [Edit the source of this page on GitHub.](https://github.com/langchain-ai/docs/edit/main/src/oss/python/integrations/tools/azure_ai_services.mdx)
</Callout>
<Tip icon="terminal" iconType="regular">
    [Connect these docs programmatically](/use-these-docs) to Claude, VSCode, and more via MCP for    real-time answers.
</Tip>
⌘I