Gradient는 간단한 web API를 통해 LLM을 fine tune하고 completion을 얻을 수 있게 해줍니다. 이 노트북은 Gradient와 함께 LangChain을 사용하는 방법을 다룹니다.

​
Imports

from langchain.chains import LLMChain
from langchain_community.llms import GradientLLM
from langchain_core.prompts import PromptTemplate

​
Environment API Key 설정

Gradient AI에서 API key를 받아야 합니다. 다양한 모델을 테스트하고 fine-tune할 수 있는 $10의 무료 크레딧이 제공됩니다. 
import os
from getpass import getpass

if not os.environ.get("GRADIENT_ACCESS_TOKEN", None):
    # Access token under https://auth.gradient.ai/select-workspace
    os.environ["GRADIENT_ACCESS_TOKEN"] = getpass("gradient.ai access token:")
if not os.environ.get("GRADIENT_WORKSPACE_ID", None):
    # `ID` listed in `$ gradient workspace list`
    # also displayed after login at at https://auth.gradient.ai/select-workspace
    os.environ["GRADIENT_WORKSPACE_ID"] = getpass("gradient.ai workspace id:")
선택사항: gradientai Python package를 사용하여 Environment 변수 GRADIENT_ACCESS_TOKEN과 GRADIENT_WORKSPACE_ID를 검증하여 현재 배포된 모델을 확인할 수 있습니다. 
pip install -qU  gradientai

Requirement already satisfied: gradientai in /home/michi/.venv/lib/python3.10/site-packages (1.0.0)
Requirement already satisfied: aenum>=3.1.11 in /home/michi/.venv/lib/python3.10/site-packages (from gradientai) (3.1.15)
Requirement already satisfied: pydantic<2.0.0,>=1.10.5 in /home/michi/.venv/lib/python3.10/site-packages (from gradientai) (1.10.12)
Requirement already satisfied: python-dateutil>=2.8.2 in /home/michi/.venv/lib/python3.10/site-packages (from gradientai) (2.8.2)
Requirement already satisfied: urllib3>=1.25.3 in /home/michi/.venv/lib/python3.10/site-packages (from gradientai) (1.26.16)
Requirement already satisfied: typing-extensions>=4.2.0 in /home/michi/.venv/lib/python3.10/site-packages (from pydantic<2.0.0,>=1.10.5->gradientai) (4.5.0)
Requirement already satisfied: six>=1.5 in /home/michi/.venv/lib/python3.10/site-packages (from python-dateutil>=2.8.2->gradientai) (1.16.0)

import gradientai

client = gradientai.Gradient()

models = client.list_models(only_base=True)
for model in models:
    print(model.id)

99148c6d-c2a0-4fbe-a4a7-e7c05bdb8a09_base_ml_model
f0b97d96-51a8-4040-8b22-7940ee1fa24e_base_ml_model
cc2dafce-9e6e-4a23-a918-cad6ba89e42e_base_ml_model

new_model = models[-1].create_model_adapter(name="my_model_adapter")
new_model.id, new_model.name

('674119b5-f19e-4856-add2-767ae7f7d7ef_model_adapter', 'my_model_adapter')

​
Gradient instance 생성

model, max_tokens 생성 수, temperature 등 다양한 parameter를 지정할 수 있습니다. 나중에 모델을 fine-tune하려고 하므로, id가 674119b5-f19e-4856-add2-767ae7f7d7ef_model_adapter인 model_adapter를 선택하지만, 어떤 base 또는 fine-tunable 모델이든 사용할 수 있습니다. 
llm = GradientLLM(
    # `ID` listed in `$ gradient model list`
    model="674119b5-f19e-4856-add2-767ae7f7d7ef_model_adapter",
    # # optional: set new credentials, they default to environment variables
    # gradient_workspace_id=os.environ["GRADIENT_WORKSPACE_ID"],
    # gradient_access_token=os.environ["GRADIENT_ACCESS_TOKEN"],
    model_kwargs=dict(max_generated_token_count=128),
)

​
Prompt Template 생성

질문과 답변을 위한 prompt template을 생성합니다. 
template = """Question: {question}

Answer: """

prompt = PromptTemplate.from_template(template)

​
LLMChain 초기화

llm_chain = LLMChain(prompt=prompt, llm=llm)

​
LLMChain 실행

질문을 제공하고 LLMChain을 실행합니다. 
question = "What NFL team won the Super Bowl in 1994?"

llm_chain.run(question=question)

'\nThe San Francisco 49ers won the Super Bowl in 1994.'

​
fine-tuning으로 결과 개선하기 (선택사항)

음 - 이것은 틀렸습니다 - San Francisco 49ers는 우승하지 않았습니다. 질문에 대한 올바른 답변은 The Dallas Cowboys!입니다. PromptTemplate을 사용하여 올바른 답변으로 fine-tuning함으로써 정확한 답변의 확률을 높여봅시다. 
dataset = [
    {
        "inputs": template.format(question="What NFL team won the Super Bowl in 1994?")
        + " The Dallas Cowboys!"
    }
]
dataset

[{'inputs': 'Question: What NFL team won the Super Bowl in 1994?\n\nAnswer:  The Dallas Cowboys!'}]

new_model.fine_tune(samples=dataset)

FineTuneResponse(number_of_trainable_tokens=27, sum_loss=78.17996)

# we can keep the llm_chain, as the registered model just got refreshed on the gradient.ai servers.
llm_chain.run(question=question)

'The Dallas Cowboys'
Edit the source of this page on GitHub.
Connect these docs programmatically to Claude, VSCode, and more via MCP for real-time answers.
⌘I