Prerequisite
Hardware Requirement
Advantech AIR-520
4U Edge AI Server powered by AMD EPYC 7003 series processor, provides up-to 64 cores, 768GB DRAM, four PCle x16 expansion slots.
-
Server grade performance powered by AMD EPYC 7003 processor -
Up-to 768GB DDR4 3200 DRAM -
Four PCIe x16 Expansion slots -
Integrated 1200W PSU for total 700W graphic card TDP
NVIDIA GPU Supported
| Product Name | Bus | Memory |
|---|---|---|
| H100 | PCIe 4.0 x16 | 80GB, HBM2e, 5120bit |
| RTX A6000 | PCIe 4.0 x16 | 48 GB, GDDR6, 384 bit |
| RTX A5000 | PCIe 4.0 x16 | 24 GB, GDDR6, 384 bit |
| GeForce RTX 4090 ( | PCIe 4.0 x16 | 24 GB, GDDR6X, 384 bit |
| L40 | PCIe 4.0 x16 | 48 GB, GDDR6, 384 bit |
| L40S | PCIe 4.0 x16 | 48 GB, GDDR6, 384 bit |
| RTX 6000 Ada Generation | PCIe 4.0 x16 | 48 GB, GDDR6, 384 bit |
| GeForce RTX 4090 D ( | PCIe 4.0 x16 | 24 GB, GDDR6X, 384 bit |
| RTX 4000 Ada Generation | PCIe 4.0 x16 | 20 GB, GDDR6, 160 bit |
| RTX 4000 SFF Ada Generation | PCIe 4.0 x16 | 20 GB, GDDR6, 160 bit |
| RTX 5000 Ada Generation | PCIe 4.0 x16 | 32 GB, GDDR6, 256 bit |
SQ Flash AI SSD (M.2)
-
SQ Flash AI SSD (M.2), 1T -
SQ Flash AI SSD (M.2), 2T
System Configuration Suggestions Based on Different Model Sizes (Full Parameters)
| Model Size | AI100E (NVMe Gen4 x 4) | GPU (Reference) | System Memory |
|---|---|---|---|
| Up to 34B | 1TB * 2 | RTX 4000 Ada x2 RTX 5000 Ada x2 RTX 6000 Ada x2 | 128GB |
| Up to 70B | 2TB * 2 | RTX 4000 Ada x4 RTX 5000 Ada x4 RTX 6000 Ada x4 | 128GB |
Hard Drive
-
4TB or 8TB SSD for Installation and Model Training
System Requirement
-
Ubuntu 22.04: - Ubuntu versions 22.04.3 and 22.04.4 were tested successfully. Generally, any 22.04.X version, where X is 3 or higher, should work without issues. The Desktop edition is recommended
-
GPU Driver: - NVIDIA driver version 535 installation.
danger
It is crucial to maintain a fixed GPU driver version. Avoid installing a newer version, as it may cause fine-tuning tasks to fail
- Installer Size (.run):
- 30G ~ 35G
- Application Footprint:
- 60G ~ 70G
info
The GenAI Studio installation file is approximately 30GB. To ensure smooth system installation, we recommend having at least 100GB of free disk space.