Prerequisite
Hardware Requirement
Advantech AIR-520
4U Edge AI Server powered by AMD EPYC 7003 series processor, provides up-to 64 cores, 768GB DRAM, four PCle x16 expansion slots
Server grade performance powered by AMD EPYC 7003 processor
Up-to 768GB DDR4 3200 DRAM
Four PCIe x16 Expansion slots
Integrated 1200W PSU for total 700W graphic card TDP
NVIDIA GPU Supported
Product Name
Bus
Memory
H100
PCIe 4.0 x16
80GB, HBM2e, 5120bit
RTX A6000
PCIe 4.0 x16
48 GB, GDDR6, 384 bit
RTX A5000
PCIe 4.0 x16
24 GB, GDDR6, 384 bit
GeForce RTX 4090 (With size restrictions)
PCIe 4.0 x16
24 GB, GDDR6X, 384 bit
L40
PCIe 4.0 x16
48 GB, GDDR6, 384 bit
L40S
PCIe 4.0 x16
48 GB, GDDR6, 384 bit
RTX 6000 Ada Generation
PCIe 4.0 x16
48 GB, GDDR6, 384 bit
GeForce RTX 4090 D (With size restrictions)
PCIe 4.0 x16
24 GB, GDDR6X, 384 bit
RTX 4000 Ada Generation
PCIe 4.0 x16
20 GB, GDDR6, 160 bit
RTX 4000 SFF Ada Generation
PCIe 4.0 x16
20 GB, GDDR6, 160 bit
RTX 5000 Ada Generation
PCIe 4.0 x16
32 GB, GDDR6, 256 bit
SQ Flash AI SSD (M.2)
SQ Flash AI SSD (M.2), 1T
SQ Flash AI SSD (M.2), 2T
System Configuration Suggestions Based on Different Model Sizes (Full Parameters)
Up to 34B
1TB * 2
RTX 4000 ada *2
RTX 5000 ada *2
RTX 6000 ada *2
128GB
Up to 70B
2TB * 2
RTX 4000 ada *4
RTX 5000 ada *4
RTX 6000 ada *4
128GB
Hard Drive
4TB or 8TB SSD for Installation and Model Training
System Requirement
Ubuntu 22.04:
Ubuntu versions 22.04.3 and 22.04.4 were tested successfully. Generally, any 22.04.X version, where X is 3 or higher, should work without issues. The Desktop edition is recommended
GPU Driver:
NVIDIA driver version 535 installation.
It is crucial to maintain a fixed GPU driver version. Avoid installing a newer version, as it may cause fine-tuning tasks to fail
Installer Size (.run):
30G ~ 35G
Application Footprint:
60G ~ 70G
Last updated