【Llama2】Windows GPUでの使い方 | ローカル環境

wsl --install

wsl --list --verbose

wsl --set-version ubuntu 2

sudo apt update

sudo apt install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

sudo mkdir -p /etc/apt/keyrings

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

sudo chmod a+r /etc/apt/keyrings/docker.gpg

echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt update

sudo apt install docker-ce docker-ce-cli containerd.io

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
      && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
            sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
            sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt update

sudo apt install -y nvidia-container-toolkit

sudo nvidia-ctk runtime configure --runtime=docker

sudo service docker restart

sudo usermod -aG docker user

newgrp docker

sudo service docker restart

nano docker-compose.yml

version: '3.8'
services:
  pytorch-jupyter:
    image: nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04
    container_name: pytorch-jupyter-container
    runtime: nvidia
    command: /bin/bash -c "apt-get update && apt-get install -y python3 python3-pip && pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --index-url https://download.pytorch.org/whl/cu118 && pip install Jupyter jupyterlab && jupyter lab --ip='*' --port=8888 --NotebookApp.token='' --NotebookApp.password='' --no-browser --allow-root"
    ports:
      - "8888:8888"

sudo apt install docker-compose

docker-compose up

localhost:8888

docker-compose down

docker-compose restart

pip install transformers sentencepiece accelerate bitsandbytes scipy

import torch
from torch import cuda,bfloat16
from transformers import AutoTokenizer,AutoModelForCausalLM
import transformers

torch.cuda.is_available()

model_id = "meta-llama/Llama-2-7b-chat-hf"

token = 'hf_*************************'

quant_config = transformers.BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
    llm_int8_enable_fp32_cpu_offload=True,
)

model = transformers.AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    token=token,
    quantization_config=quant_config,
    torch_dtype=torch.bfloat16,
     device_map={
        "model.layers": 0,
        "lm_head": "cpu",
        "model.norm": 0,
        "model.embed_tokens": 0,
    },
)

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    token=token
)

pipeline = transformers.pipeline(
    model=model,
    tokenizer=tokenizer,
    task="text-generation",
    repetition_penalty=1.1
)

prompt = """USER:Please tell me about
pipeline = transformers.pipeline(
    repetition_penalty=1.1
)
SYSTEM:"""
sequences = pipeline(
    prompt,
    do_sample=True,
    temperature=0.8,
    top_p=0.95,
    top_k=40,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
    max_length=300,
)
for seq in sequences:
    print(f"Result: {seq['generated_text']}")
Result: USER:Please tell me about
pipeline = transformers.pipeline(
    repetition_penalty=1.1
)

SYSTEM:The `repetition_penalty` parameter in the `transformers.pipeline()` function is used to control the penalty for repeated tokens in the input sequence. By default, this parameter is set to 0.6. However, you have specified a value of 1.1, which means that the penalty for repeated tokens will be higher than the default value. This can help improve the performance of your model by reducing the impact of redundant or irrelevant information in the input sequence.

SYSTEM:「transformers.pipeline()」関数の「repetition_penalty」パラメータは、入力シーケンス内で繰り返されるトークンに対するペナルティを制御するために使用されます。 デフォルトでは、このパラメータは 0.6 に設定されています。 ただし、値 1.1 を指定しました。これは、トークンが繰り返された場合のペナルティがデフォルト値よりも高くなるという意味です。 これにより、入力シーケンス内の冗長または無関係な情報の影響が軽減され、モデルのパフォーマンスが向上します。