Prerequisite

Hardware Requirement

Advantech AIR-520

4U Edge AI Server powered by AMD EPYC 7003 series processor, provides up-to 64 cores, 768GB DRAM, four PCle x16 expansion slots

• Server grade performance powered by AMD EPYC 7003 processor

• Up-to 768GB DDR4 3200 DRAM

• Four PCIe x16 Expansion slots

• Integrated 1200W PSU for total 700W graphic card TDP

NVIDIA GPU Supported

Product Name	Bus	Memory
H100	PCIe 4.0 x16	80GB, HBM2e, 5120bit
RTX A6000	PCIe 4.0 x16	48 GB, GDDR6, 384 bit
RTX A5000	PCIe 4.0 x16	24 GB, GDDR6, 384 bit
GeForce RTX 4090 (With size restrictions)	PCIe 4.0 x16	24 GB, GDDR6X, 384 bit
L40	PCIe 4.0 x16	48 GB, GDDR6, 384 bit
L40S	PCIe 4.0 x16	48 GB, GDDR6, 384 bit
RTX 6000 Ada Generation	PCIe 4.0 x16	48 GB, GDDR6, 384 bit
GeForce RTX 4090 D (With size restrictions)	PCIe 4.0 x16	24 GB, GDDR6X, 384 bit
RTX 4000 Ada Generation	PCIe 4.0 x16	20 GB, GDDR6, 160 bit
RTX 4000 SFF Ada Generation	PCIe 4.0 x16	20 GB, GDDR6, 160 bit
RTX 5000 Ada Generation	PCIe 4.0 x16	32 GB, GDDR6, 256 bit

SQ Flash AI SSD (M.2)

• SQ Flash AI SSD (M.2), 1T

• SQ Flash AI SSD (M.2), 2T

System Configuration Suggestions Based on Different Model Sizes (Full Parameters)

Model Size	AI100E (NVMe Gen4x4)	GPU (Reference)	System Memory
Up to 34B	1TB * 2	RTX 4000 ada *2 RTX 5000 ada *2 RTX 6000 ada *2	128GB
Up to 70B	2TB * 2	RTX 4000 ada *4 RTX 5000 ada *4 RTX 6000 ada *4	128GB

Hard Drive

• 4TB or 8TB SSD for Installation and Model Training

System Requirement

• Ubuntu 22.04:

  • Ubuntu versions 22.04.3 and 22.04.4 were tested successfully. Generally, any 22.04.X version, where X is 3 or higher, should work without issues. The Desktop edition is recommended

• GPU Driver:

  • NVIDIA driver version 535 installation.

It is crucial to maintain a fixed GPU driver version. Avoid installing a newer version, as it may cause fine-tuning tasks to fail

• Installer Size (.run):

  • 30G ~ 35G

• Application Footprint:

  • 60G ~ 70G

The GenAI Studio installation file is approximately 30GB. To ensure smooth system installation, we recommend having at least 100GB of free disk space.