Train Your Own ChatGPT Model for $75 with Nanochat on Hyperstack

Want to build your own ChatGPT-like model? Sounds fascinating until you start worrying about costs, hardware limitations and setup headaches. Thanks to AI expert Andrej Karpathy and his open-source repo Nanochat, you can now fine-tune a full conversational model on your own hardware.

While Andrej mentions the training cost of around $100, you can do the same for just $75 with Hyperstack’s high-performance NVIDIA H100 PCIe NVLink GPUs.

In our latest tutorial, we show you exactly how to make it happen, step by step.

1. Go to the Hyperstack website and log in to your account.
2. If you're new to Hyperstack, you'll need to create an account and set up your billing information. Check our documentation to get started with Hyperstack.
3. Once logged in, you'll be greeted by the Hyperstack dashboard, which provides an overview of your resources and deployments.

Step 2: Deploying a New Virtual Machine

Initiate Deployment

1. Look for the "Deploy New Virtual Machine" button on the dashboard.
2. Click it to start the deployment process.

Select Hardware Configuration

1. In the hardware options, choose the "8xH100 PCIe NVLink" flavour.

Choose the Operating System

1. Select the "Ubuntu Server 24.04 LTS R570 CUDA 12.8 with Docker".

Select a keypair

1. Select one of the key pairs in your account. Don't have a keypair yet? See our Getting Started tutorial for creating one.

Network Configuration

1. Ensure you assign a Public IP to your Virtual machine.
2. This allows you to access your VM from the internet, which is crucial for remote management and API access.

Enable SSH Access

1. Make sure to enable an SSH connection.
2. You'll need this to securely connect and manage your VM.

Review and Deploy

1. Double-check all your settings.
2. Click the "Deploy" button to launch your virtual machine.

Step 3: Accessing Your VM

Once the initialisation is complete, you can access your VM:

Locate SSH Details

1. In the Hyperstack dashboard, find your VM's details.
2. Look for the public IP address, which you will need to connect to your VM with SSH.

Connect via SSH

1. Open a terminal on your local machine.
2. Use the command ssh -i [path_to_ssh_key] [os_username]@[vm_ip_address] (e.g: ssh -i /users/username/downloads/keypair_hyperstack ubuntu@0.0.0.0.0)
3. Replace username and ip_address with the details provided by Hyperstack.

Step 4: Download and Prepare the Script

Click here to download the script run_nanochat.sh.

Step 5: Start Training

chmod +x ./run_nanochat.sh
screen -L -Logfile nanochat.log -S nanochat bash run_nanochat.sh

python -m scripts.chat_cli -p "Why is the sky blue?"

python -m scripts.chat_web

1. Open your VM's firewall settings.
2. Allow port 8000 for your IP address (or leave it open to all IPs, though this is less secure and not recommended). For instructions, see here.
3. Visit http://[public-ip]:8000 in your browser. For example: http://198.145.126.7:3000
4. Set up an admin account for OpenWebUI and save your username and password for future logins. See the attached screenshot.

And voila, you can start talking to your fine-tuned model!

When you're finished with your current workload, you can hibernate your VM to avoid incurring unnecessary costs:

1. In the Hyperstack dashboard, locate your Virtual machine.
2. Look for a "Hibernate" option.
3. Click to hibernate the VM, which will stop billing for compute resources while preserving your setup.

To continue your work without repeating the setup process:

1. Return to the Hyperstack dashboard and find your hibernated VM.
2. Select the "Resume" or "Start" option.
3. Wait a few moments for the VM to become active.
4. Reconnect via SSH using the same credentials as before.

New to Hyperstack? Sign up on Hyperstack Today to Get Started.

FAQs

What is Nanochat?

Nanochat is an open-source project by Andrej Karpathy that shows how to train a small ChatGPT-like model from scratch. 

What can I do with Nanochat?

You can use Nanochat to train your own conversational AI model, experiment with fine-tuning, and learn the inner workings of transformer-based models, all using just a few hundred lines of code.

How much does it cost to run Nanochat on Hyperstack?

On Hyperstack, you can train and run Nanochat for $75, thanks to affordable on-demand access to high-performance GPUs like the NVIDIA H100 PCIe NVLink.

Which GPUs should I use to train Nanochat?

For best performance, use 8× NVIDIA H100 PCIe NVLink GPUs with Ubuntu on Hyperstack.