Samsung’s IT services arm and other companies are said to be testing out a processor that sports more than 1,000 general-purpose RISC-V cores to deliver what the chip’s designer claims is faster and more energy-efficient AI inference performance than power-hungry specialty silicon.
The chip designer, Esperanto Technologies, said Thursday Samsung SDS and other unnamed companies, which it only identified as “lead customers,” are doing initial evaluations of the startup’s ET-SoC-1 AI inference accelerator.
The Mountain View, California-based startup was founded in 2014 by Dave Ditzel, a semiconductor industry veteran who worked on parallel computing architectures at Intel and, earlier in his career, led the development of the SPARC CPU instruction set architecture at Sun Microsystems. Esperanto is now led by Art Swift, who has held several semiconductor leadership roles, including CEO of Wave Computing.
The ET-SoC-1 packs 1,088 energy-efficient, 64-bit processor cores that use the RISC-V instruction set architecture, an emerging alternative to the x86 and Arm ISAs. These cores are accompanied by their own vector/tensor math units for acceleration machine-learning operations, and the chip also comes with four high-performance RISC-V cores, plus more than 160 million bytes of on-chip SRAM (more than 152MB, then) and interfaces for flash memory and external DRAM.
With this architecture, Esperanto claims it has the fastest AI RISC-V chip. Combined with its low-power requirements, the startup said the ET-SoC-1 was “designed to meet the performance, power and total cost of ownership requirements of large-scale data center customers.”
While Esperanto claims the ET-SoC-1 can “run any machine learning workload well,” the company said the chip excels at ML recommendation, one of the most common server applications run by so-called hyperscalers like Facebook parent company Meta and Amazon.
Plaudits from Samsung SDS
Patrick Bangert, vice president of AI at Samsung SDS, said his data science team “was very impressed” with the company’s first test of the ET-SoC-1. He added that processors from Esperanto’s competitors do not offer the same level of performance scaling as the ET-SoC-1, which means that the chip does a good job at providing a higher level of performance in proportion to the amount of chips used.
“It was fast, performant and overall easy to use. In addition, the SoC demonstrated near-linear performance scaling across different configurations of AI compute clusters. This is a capability that is quite unique, and one we have yet to see consistently delivered by established companies offering alternative solutions to Esperanto,” Bangert said in a quote provided by Esperanto.
This is quite the testimony from a major IT provider, which sells a variety of AI and data analytics services through its Brightics AI software platform.
But it’s one thing to sample a new chip. It’s another thing entirely for a large company to make a large-volume purchase order and rely on such chips to power revenue-generating products and services. Thus, Esperanto’s sampling news is but one of many milestones it will need to achieve before causing concern for the dominant provider of AI chips, Nvidia, and other chip designers.
As such, Esperanto said it’s looking for other companies that want to join its evaluation program, which allows users to rest the ET-SoC-1 on a variety of off-the-shelf AI models, including recommendation, transformer and visual network models. The program lets users test various models, data types, batch sizes and compute configurations of up to 32 clusters.
Good software support, but is it too late?
Karl Freund, principal analyst at Cambrian-AI Research, said Esperanto demonstrated “rock solid” performance for ET-SoC-1 with the ResNet 50, DLRM and Transformer models, though he can’t share the results yet. He added that he expects the chip to only require 20 watts to run at full power.
Freund said he was initially skeptical that the ET-SoC-1 could provide a high level of inference performance using general-purpose RISC-V cores, but the results proved him wrong. “What really makes this approach unique, is that the RISC-V cores are actually doing the heavy lifting, not offloading the matrix multiplies to a MAC core or a GPU,” he said.
Just as important, Freund said, Esperanto “has the programming tools and software stack to more easily adapt to new AI workloads, alongside non-AI workloads, all running on the same silicon.”
However, Freund admitted that Esperanto’s chip has arrived a “bit late,” so there is a question of whether the startup can keep up with other companies working on low-power chips for inference. ®