Prerequisite
Hardware Requirement
Advantech AIR-520
4U Edge AI Server powered by AMD EPYC 7003 series processor, provides up-to 64 cores, 768GB DRAM, four PCle x16 expansion slots.
- Server grade performance powered by AMD EPYC 7003 processor
- Up-to 768GB DDR4 3200 DRAM
- Four PCIe x16 Expansion slots
- Integrated 1200W PSU for total 700W graphic card TDP
NVIDIA GPU Supported
| Product Name | Bus | Memory |
|---|---|---|
| H100 | PCIe 4.0 x16 | 80GB, HBM2e, 5120bit |
| RTX A6000 | PCIe 4.0 x16 | 48 GB, GDDR6, 384 bit |
| RTX A5000 | PCIe 4.0 x16 | 24 GB, GDDR6, 384 bit |
| GeForce RTX 4090 (With size restrictions) | PCIe 4.0 x16 | 24 GB, GDDR6X, 384 bit |
| L40 | PCIe 4.0 x16 | 48 GB, GDDR6, 384 bit |
| L40S | PCIe 4.0 x16 | 48 GB, GDDR6, 384 bit |
| RTX 6000 Ada Generation | PCIe 4.0 x16 | 48 GB, GDDR6, 384 bit |
| GeForce RTX 4090 D (With size restrictions) | PCIe 4.0 x16 | 24 GB, GDDR6X, 384 bit |
| RTX 4000 Ada Generation | PCIe 4.0 x16 | 20 GB, GDDR6, 160 bit |
| RTX 4000 SFF Ada Generation | PCIe 4.0 x16 | 20 GB, GDDR6, 160 bit |
| RTX 5000 Ada Generation | PCIe 4.0 x16 | 32 GB, GDDR6, 256 bit |
SQ Flash AI SSD (M.2)
- SQ Flash AI SSD (M.2), 1T
- SQ Flash AI SSD (M.2), 2T
System Configuration Suggestions Based on Different Model Sizes (Full Parameters)
| Model Size | AI100E (NVMe Gen4 x 4) | GPU (Reference) | System Memory |
|---|---|---|---|
| Up to 34B | 1TB * 2 | RTX 4000 Ada x2 RTX 5000 Ada x2 RTX 6000 Ada x2 | 128GB |
| Up to 70B | 2TB * 2 | RTX 4000 Ada x4 RTX 5000 Ada x4 RTX 6000 Ada x4 | 128GB |
Hard Drive
- 4TB or 8TB SSD for Installation and Model Training.
System Requirement
- Ubuntu 22.04:
- Ubuntu versions 22.04.3 and 22.04.4 were tested successfully. Generally, any 22.04.X version, where X is 3 or higher, should work without issues. The Desktop edition is recommended
- GPU Driver:
- NVIDIA driver version 550 installation.
danger
It is crucial to maintain a fixed GPU driver version. Avoid installing a newer version, as it may cause fine-tuning tasks to fail
- Installer Size (.run):
- ~ 45G
- Application Footprint:
- 60G ~ 70G
info
The GenAI Studio installation file is approximately 45GB. To ensure smooth system installation, we recommend having at least 100GB of free disk space.