AI LLM provider backed by MLPerf cofounder bets barn on mature AMD Instinct
MI GPU but where are the MI300s?

Date:
Tue, 03 Oct 2023 20:05:40 +0000

Description:
Lamini wants enterprises to buy their own LLM superstation to run LLMs in virtual PCs or on-premise.

FULL STORY ======================================================================

With demand for enterprise-grade large language models (LLMs) surging over
the last year or so, Lamini has opened the doors to its LLM Superstation powered by AMDs Instinct MI GPUs.

The firm claims its been running LLMs on over 100 AMD instinct GPUs in secret for the last year in production situations even before ChatGPT launched.
With its LLM Superstation, its opening the doors to more potential customers to run their models on its infrastructure.

These platforms are powered by AMD Instinct MI210 and MI250 accelerators, as opposed to the industry-leading Nvidia H100 GPUs which. By opting for its AMD GPUs, Lamini quips, businesses can stop worrying about the 52-week lead time. AMD vs Nvidia GPUs for LLMs

Although Nvidias GPUs including the H100 and A100 are those most commonly
in use to power LLMs such as ChatGPT, AMDs own hardware is comparable .

For example, the Instinct MI250 offers up to 362 teraflops of computing power for AI workloads, with the MI250X pushing this do 383 teraflops. The Nvidia A100 GPU, by way of contrast, offers up to 312 teraflops of computing power, according to TechRadar Pro sister site Toms Hardware .

"Using Lamini software, ROCm has achieved software parity with CUDA for LLMs, said Lamini CTO Greg Diamos, who is also the cofounder of MLPerf. We chose
the Instinct MI250 as the foundation for Lamini because it runs the biggest models that our customers demand and integrates finetuning optimizations.

We use the large HBM capacity (128GB) on MI250 to run bigger models with
lower software complexity than clusters of A100s." (Image credit: Lamini)

AMDs GPUs can, in theory, certainly compete with Nvidias. But the real crux
is availability, with systems such as Laminis LLM Superstation able to offer enterprises the opportunity to take on workloads immediately.

Theres also the question mark, however, over AMDs next-in-line GPU, the MI300 . Businesses are currently able to sample the MI300A now, while the MI300X is being sampled in the coming months.

According to Toms Hardware, the MI300X offers up to 192GB memory, which is double the H100, although we dont yet fully know what the compute performance looks like. Nevertheless, its certainly set to be comparable to the H100.
What would give Laminis LLM Superstation a real boost is building and
offering its infrastructure powered by these next-gen GPUs. More from TechRadar Pro AMD vs Nvidia: who is the graphics card champion? We've rounded up the best graphics cards for all budgets Why aren't people buying GPUs? Nvidia has the answers to its own problem



======================================================================
Link to news story: https://www.techradar.com/pro/ai-llm-provider-backed-by-mlperf-cofounder-bets- barn-on-mature-amd-instinct-mi-gpu-but-where-are-the-mi300s


--- Mystic BBS v1.12 A47 (Linux/64)
* Origin: tqwNet Technology News (1337:1/100)