Embedded & Edge Computing
Running AI with Limited RAM
Labeled RAM is not the memory a model can use. The operating system, runtime, and key-value cache all take a share. The remaining figure is what the weights must fit inside.
This note is a starting point. A published study with measurements will appear on the projects page only after the work has been completed and checked.