Esperanto Technologies Introduces First Generative AI Appliance Based on RISC-V, Enabling Developers to Easily Create and Deploy Purpose-Built Vertical Applications
Enables Fast and Secure Deployment of Fine-Tuned Business Applications
Including Summarization, Coding, Query and Image Generation
Based on the Latest Open-Source Generative AI Models
MOUNTAIN VIEW, Calif., September 12, 2023 – Esperanto Technologies™, the leading developer of high-performance, energy-efficient artificial intelligence (AI) and high-performance computing (HPC) solutions based on the RISC-V instruction set, today announced the industry’s first Generative AI Appliance based on RISC-V technology. Esperanto’s Data Science team contributed heavily to its design, targeting customers wanting to develop and deploy business applications quickly using the latest open-source Generative AI foundation models. Esperanto’s Generative AI Appliance is an integrated software/hardware solution that can be installed in private datacenters or at the enterprise edge using an industry-standard server form factor. Because it is preloaded and self-contained, it delivers high levels of data privacy and lower total cost of ownership (TCO) while eliminating the need for developers to constantly download, port and tune the latest Large Language Models (LLMs) and Diffusion Models to expensive GPU-based hardware.
Esperanto’s new appliance is ideal for organizations that want to leverage the benefits of Generative AI technology to create custom applications initially around information summarization, organizational data/knowledge query, computer code generation and translation and image generation. Esperanto’s Data Science and Software teams designed it to support various application UI and output texts, computer programs and images, and is continually expanding the availability of LLMs and Diffusion models as they are made public. Examples of industries that can benefit from Esperanto’s new solution include the healthcare and legal professions which require quick and accurate summaries of complex descriptions while maintaining data privacy, and the financial industry which can translate its legacy code base to more modern and maintainable programming languages.
To request additional details and pricing, please visit www.esperanto.ai/contact.
“Generative AI is revolutionizing the way we create and summarize content, generate and translate computer code, and generate visual and video content. However, creating and deploying LLM-based applications typically requires large teams of data scientists, long development times and expensive, hard-to-obtain GPU-based platforms. This can make Generative AI strategies impractical for most organizations today,” said Art Swift, president and CEO at Esperanto Technologies. “Esperanto recognizes these challenges and has developed its new Generative AI Appliance based on its advanced RISC-V hardware using pretrained LLMs that are highly accurate but with much faster development and strong data privacy.”
Esperanto’s Generative AI Appliance is currently running the latest LLMs and image generation models such as LLaMA 2, Vicuna, StarCoder, OpenJourney and Stable Diffusion, and the company's strategy is to continuously update the system with the latest versions of popular open-source models as soon as they are released.
“We are in the early stages of a multi-year super cycle for merchant ASICs, driven by the adoption of Generative AI, an increase in AI training, significant growth of AI inferencing, and HPC workflows,” said Ben Bajarin, CEO and principal analyst at Creative Strategies, Inc. “We are forecasting an Enterprise Edge infrastructure refresh as companies look to run more AI and HPC workloads on-prem for cost, privacy, and data sovereignty reasons. In addition, energy efficiency is a growing priority, so offerings like Esperanto’s that have a strong dollar-per-watt value are well positioned.”
“The market is trending toward smaller LLM and diffusion models – 30 billion parameters and below – driven by reducing the high cost of inference on very large models”, said Karl Freund, founder and principal analyst at Cambrian-AI Research. “These models are trained to be highly accurate with much lower training and inference costs. There is a lot of money to be made in this space, and inference solutions like Esperanto’s Generative AI Appliance should save customers significant costs versus GPU-based systems.”
Esperanto’s Generative AI Appliance is available now and includes currently available ET-SoC-1 AI Accelerator chips that can run up to 4 LLMs simultaneously. The appliance is delivered in a standard 2U-high rack-mounted chassis and is available directly from Esperanto.
To request additional details and pricing, please visit www.esperanto.ai/contact.
About Esperanto Technologies:
Esperanto Technologies develops massively parallel, high-performance, energy-efficient computing solutions for Generative AI, other AI, and massively parallel HPC workloads, based on the open standard RISC-V instruction set architecture. Esperanto is headquartered in Mountain View, California with additional engineering sites in Portland, Oregon; Austin, Texas; Barcelona, Spain; and Belgrade, Serbia. For more information, please visit https://www.esperanto.ai/
EE Times: "Esperanto Pivots to HPC and Generative AI"
Esperanto Technologies is featured in EE Times: “Esperanto Pivots to HPC and Generative AI”
By Sally Ward-Foxton
Click here to read the article at EE Times...
Esperanto featured in EE Times "Silicon 100" Startups to watch in 2023
Esperanto Technologies is featured in the EE Times Silicon 100: “Startups Worth Watching in 2023.”
By Peter Clarke:
- The Silicon 100 is the “who’s who and who’s doing what” in the global electronics and semiconductor industries.
- The Silicon 100, now in its 23rd edition, is marked by three major trends: high levels of interest in artificial intelligence, a significant surge in Chinese startup activity and an increase in startup formation globally across the breadth of the electronics and semiconductor domain.
- Peter Clarke, a technology and business journalist who has curated the EE Times Silicon 100 list since its inception in 2004, has been exploring where technical expertise meets continuous innovation and tracking the peaks and troughs of investment returns.
Purchase the full report here...
Esperanto Technologies Expands Global Partner Network
Company’s Partners Add System Integration and Solutions Expertise As Well As Enhanced Access to US, European and Japanese Markets
MOUNTAIN VIEW, Calif., May 22, 2023 – Esperanto Technologies™, the leading developer of high performance, energy-efficient artificial intelligence (AI) solutions based on the RISC-V instruction set, today announced that it is accelerating deployment of its ET-SoC-1 solutions through an expanded network of partners in key international markets. These companies enhance Esperanto’s capabilities with deep expertise in datacenter and edge systems technologies, and strengthen access to key customers in AI, high performance computing (HPC), generative AI and mixed-mode (AI plus HPC) applications.
To address the demand for AI-enabled systems, Esperanto has entered into a strategic partnership with Penguin Solutions, an SGH™ brand (Nasdaq: SGH) that provides HPC, AI, and Internet-of-Things (IoT) technologies, services, and expertise that span the continuum of edge, core, and cloud. Esperanto and Penguin Solutions are working closely to develop multiple systems in various form factors to address the demand for AI and HPC acceleration in the datacenter and at the edge.
In addition, Esperanto announced several value-added partnerships. In Italy, Esperanto announced a partnership with E4 Computer Engineering, an internationally recognized system integrator and solution provider for high-grade expertise in HPC, High Performance Data Analytics, AI and Deep Learning. In Germany, Esperanto announced a partnership with MEGWARE GmbH, one of Europe's leading supercomputing specialists. In Japan, Esperanto announced a partnership with Elematec Corporation, a provider of one-stop solutions for on-site planning and development, design and manufacturing services.
Esperanto is seeing growing demand worldwide for its energy efficient, massively parallel RISC-V solutions. Semico Research Corporation forecasts that the total market revenue for RISC-V AI SoCs will grow at 72.2% CAGR to $29 billion by 2027. Esperanto is currently serving datacenter and enterprise edge applications with its ET-SoC-1 AI/HPC Accelerator chip, featuring more than 1,000 64-bit RISC-V cores with attached vector/tensor units, with value-added card and server form factors provided by both Esperanto and its partners. In addition, Esperanto’s newly announced Cloud Access Program enables broader access to these systems for evaluation of its technology worldwide.
To request additional details and pricing, please visit www.esperanto.ai/contact.
“As one of the first companies shipping massively parallel RISC-V based hardware, Esperanto is in a unique position to serve these market needs through our expanded worldwide partner network,” said Art Swift, president and CEO at Esperanto Technologies. “We’ve been shipping RISC-V evaluation servers globally since last year, and now our expanded partner network will enable Esperanto to reach more customers interested in RISC-V acceleration for AI, HPC and mixed-mode workloads.”
“Penguin helps to accelerate customer access to the latest technology innovations for both AI and high performance computing,” said Thierry Pellegrino, senior vice president at SGH and president of Intelligent Platform Solutions. “By partnering closely with Esperanto, we are able to offer one of the industry’s leading RISC-V solutions in a variety of form factors for customers spanning data center to edge markets. Our valued partnership with Esperanto will help us continue to innovate in our HPC and AI solution designs and keep pace with the growing demand for advancements in generative AI, machine learning, and data analytics.”
“At E4, we are seeing significant interest among our customer base in technical solutions based on RISC-V,” said Cosimo Gianfreda, CEO at E4 Computer Engineering. “Our partnership with Esperanto enables us to bring the latest in RISC-V-based AI and HPC acceleration into systems that we deliver to the most advanced enterprises, universities and centers for advanced research and computing.”
“MEGWARE delivers high-performance computers to research institutes, universities, and industrial and business customers throughout Europe,” said Axel Auweter, CTO at MEGWARE GmbH. “With the growth of RISC-V in the European HPC space, we are excited about our partnership with Esperanto which will enable us to bring AI and HPC acceleration based on RISC-V to the public and industrial customers that we serve.”
“Elematec delivers products and services to a broad set of customers across digital electronics, the automotive industry and industrial and home segments,” said Akira Yokode, CEO and Chairman at Elematec. “We see great opportunities for RISC-V AI acceleration across our broad customer base, and we are very enthusiastic about partnering with Esperanto Technologies to address these opportunities through our multi-pronged business model.”
About Esperanto Technologies:
Esperanto Technologies develops massively parallel, high-performance, energy-efficient computing solutions for Artificial Intelligence / Machine Learning and massively parallel general purpose workloads, based on the open standard RISC-V instruction set architecture. Esperanto is headquartered in Mountain View, California with additional engineering sites in Portland, Oregon; Austin, Texas; Barcelona, Spain; and Belgrade, Serbia. For more information, please visit https://www.esperanto.ai/
Esperanto Technologies Launches General Purpose SDK Enabling Customers to Accelerate Parallelized HPC Workloads
Customers Can Now Directly Program ET-SoC-1 using Standard C or C++ to Accelerate Non-AI Workloads Using Over 1,000 64-bit RISC Cores on a Single Chip
MOUNTAIN VIEW, Calif., May 1, 2023 – Esperanto Technologies™, the leading developer of high performance, energy-efficient artificial intelligence (AI) solutions based on the RISC-V instruction set, today introduced a new software development kit (SDK) that extends support beyond AI inference to a broader range of general purpose, high-performance computing (HPC) and mixed HPC plus AI use cases. The new SDK complements Esperanto’s existing AI-centric software stack to enable direct programming of each of ET-SoC-1’s 1,000+ 64-bit RISC-V CPUs and their accompanying vector/tensor units for massively parallel computation.
“One of Esperanto’s key strengths in the AI acceleration market segment is that we use standard 64-bit RISC-V cores as the basis for our compute fabric, rather than overly specialized hardware. Many customers are keenly interested in directly programming these cores in addition to using our ML software stack to accelerate machine learning workloads,” said Art Swift, president and CEO at Esperanto Technologies. “We are excited to make this SDK available so that customers can bring their own parallel workloads to ET-SoC-1 and take advantage of its 1,000+ RISC-V cores for massive parallelism.”
Esperanto’s General Purpose SDK enables the direct programming of all of ET-SoC-1’s 64-bit RISC-V cores and their attached vector/tensor units to accelerate mathematical computation. This makes ET-SoC-1 ideal for accelerating digital signal processing and other highly parallel workloads. Customers can also combine machine learning and general purpose workloads on the same ET-SoC-1 chip, enabling pre- and post-processing of AI workload data on the same device.
The General Purpose SDK is incorporated into Esperanto’s on-prem and Cloud Access evaluation systems, along with Esperanto’s current machine learning (ML) SDK, and a suite of performance analysis tools and pre-integrated models for fast experimentation, benchmarking and development. Esperanto evaluation servers deliver high performance combined with high energy efficiency and low total cost of ownership (TCO). Available in a standard 2U-high form factor, each Esperanto evaluation server includes dual Xeon host processors and either 8 or 16 ET-SoC-1 PCIe cards. Each Esperanto PCIe card has over 1,000 64-bit RISC-V CPUs with attached vector/tensor units, delivering up to over 16,000 RISC-V CPUs per server. Esperanto’s evaluation servers enable customers to obtain performance and power data from running a variety of industry standard AI models, including Generative AI, as well as the ability to bring their own models and data.
To request additional details and pricing, please visit esperanto.ai/technology/#eap.
About Esperanto Technologies:
Esperanto Technologies develops massively parallel, high-performance, energy-efficient computing solutions for Artificial Intelligence / Machine Learning and massively parallel general purpose workloads, based on the open standard RISC-V instruction set architecture. Esperanto is headquartered in Mountain View, California with additional engineering sites in Portland, Oregon; Austin, Texas; Barcelona, Spain; and Belgrade, Serbia. For more information, please visit https://www.esperanto.ai/
Esperanto Technologies Launches New Cloud Access Program to Broaden Access to its Massively Parallel, Low Power RISC-V Solutions
New Offering Serves Growing Customer Interest and Complements Current System Shipments
MOUNTAIN VIEW, Calif., April 24, 2023 – Esperanto Technologies™, the leading developer of high performance, energy-efficient artificial intelligence solutions based on the RISC-V instruction set, today announced the availability of a new cloud-based program that expands access to the company’s fully configured evaluation servers featuring its industry leading ET-SoC-1 compute accelerators.
Esperanto has been shipping fully configured evaluation servers globally for on-premises evaluations, becoming one of the first companies in the industry to deliver massively parallel, low-power RISC-V based systems directly to customers. To address increasing market interest, the company is now expanding access to the same fully configured evaluation servers to a broader set of customers and developers via the cloud.
To request additional details and pricing, please visit www.esperanto.ai/contact.
“With the support of our expanding network of partners, Esperanto has been shipping evaluation systems to customers in the U.S., Asia and Europe since last year. Our new cloud evaluation service offers remote access to the same hardware and software stacks as the on-prem evaluation systems, enabling us to serve more customers in a broader range of markets more quickly,” said Art Swift, president and CEO at Esperanto Technologies. “As demand for alternatives to existing higher cost, higher power offerings accelerates, we are ramping availability to serve those customers and speed their transition to RISC-V.”
Esperanto’s AI evaluation servers deliver high performance combined with high energy efficiency and low total cost of ownership (TCO). With a 2U-high form factor that slides into standard datacenter racks, each Esperanto server comes with dual Xeon host processors and either 8 or 16 ET-SoC-1 cards. Each card has over 1,000 64-bit RISC-V CPUs with attached vector/tensor units, delivering up to over 16,000 CPUs per server. A fully configured datacenter rack with 20 Esperanto servers can deliver up to ~320,000 64-bit RISC-V CPUs.
Esperanto’s evaluation servers enable customers to obtain performance and power data from running a variety of industry standard AI models including generative AI, transformers, computer vision models and recommendation systems, as well as bring their own models and data.
About Esperanto Technologies:
Esperanto Technologies develops massively parallel, high-performance, energy-efficient computing solutions for Artificial Intelligence / Machine Learning and massively parallel general purpose workloads, based on the open standard RISC-V instruction set architecture. Esperanto is headquartered in Mountain View, California with additional engineering sites in Portland, Oregon; Austin, Texas; Barcelona, Spain; and Belgrade, Serbia. For more information, please visit https://www.esperanto.ai/
Esperanto Technologies Announces RISC-V Industry Milestone Of Generative AI Models Running on ET-SoC-1; Access to be Made Available to the RISC-V Research Community
Initial Work Includes Running a Range of Large Language Models,
Including Meta’s Open Pre-Trained Transformer Generative AI Model,
Running as Low as 25W for AI Inferencing on ET-SoC-1
MOUNTAIN VIEW, Calif., April 19, 2023 – Esperanto Technologies™, the leading developer of high performance, energy-efficient artificial intelligence solutions based on the RISC-V instruction set, today announced that it has ported and is running a range of Generative AI models on its low power RISC-V hardware. Esperanto plans to provide access to researchers in the RISC-V community as part of the company’s mission to help “democratize AI” for the broader industry and help accelerate development of Generative AI technology on RISC-V.
Researchers interested in obtaining more information can contact Esperanto at www.esperanto.ai/contact.
This development is part of Esperanto’s strategy to extend the benefits of RISC-V technology in AI and general-purpose applications from cloud to edge. Areas of focus for Esperanto’s Generative AI efforts include use cases for large language models (LLMs) where using low power hardware with improved total cost of ownership (TCO) compared to existing offerings are key.
In particular, several versions of Meta’s Open Pre-Trained Transformer (OPT) model are now running on Esperanto’s hardware at multiple precision levels and context sizes with power levels as low as 25W per chip for inferencing. The rapid porting and bring-up of the OPT models onto ET-SoC-1 silicon were enabled by Esperanto’s machine learning software development kit, which is currently in use by the company’s commercial customers. Researchers who are granted access to Esperanto’s solutions for R&D purposes will be required to comply with Meta’s open-source license as well as other program terms.
“Generative AI is one of the latest advancements in machine learning, and we are pleased to contribute elements of our efforts in the area of large language models to the RISC-V research community,” said Art Swift, president and CEO at Esperanto Technologies.
“Penguin Computing is proud to be a strategic partner of Esperanto, helping to accelerate the deployment of high performance, low power Generative AI solutions,” said Thierry Pellegrino, president of Penguin Solutions and senior vice president of Intelligent Platform Solutions at Smart Global Holdings. “We are excited to be working with Esperanto to extend the deployment of its RISC-V solutions to a broader set of customers that are searching for low power AI inference solutions and reduced total cost of ownership.”
“The growing interest in RISC-V signals an important inflection point for the semiconductor industry. This development in the area of Generative AI will further accelerate the evaluation of RISC-V by researchers looking for breakthroughs in this exciting field, and by developers seeking to increase their competitive differentiation beyond what existing platforms can provide,” said Ben Bajarin, CEO and principal analyst at Creative Strategies.
“RISC-V offers unparalleled opportunities for collaboration and customization, making it ideally suited for this next wave of AI innovation,” said Calista Redmond, CEO of RISC-V International. “Esperanto is one of the companies leading the charge in this space, pushing the limits of performance and power-efficiency to make Generative AI development more accessible.”
For commercial customers, Esperanto is currently shipping AI evaluation servers which deliver high performance combined with high energy efficiency and low TCO. Available in a standard 2U-high form factor, each Esperanto evaluation server includes dual Xeon host processors and either 8 or 16 ET-SoC-1 PCIe cards. Each Esperanto PCIe card has over 1,000 64-bit RISC-V CPUs with attached vector/tensor units, delivering up to 16,000 RISC-V CPUs per server. Esperanto’s evaluation servers enable customers to obtain performance and power data from running a variety of industry standard AI models, as well as the ability to bring their own models and data.
About Esperanto Technologies:
Esperanto Technologies develops massively parallel, high-performance, energy-efficient computing solutions for Artificial Intelligence / Machine Learning and massively parallel general purpose workloads, based on the open standard RISC-V instruction set architecture. Esperanto is headquartered in Mountain View, California with additional engineering sites in Portland, Oregon; Austin, Texas; Barcelona, Spain; and Belgrade, Serbia. For more information, please visit https://www.esperanto.ai/
Esperanto Presents at the RISC-V Summit 2022
Hear Dave Ditzel of Esperanto Technologies speak at the RISC-V Summit 2022
Dave Ditzel, Founder and CTO of Esperanto Technologies, will deliver a presentation, “Real World Results using Thousands of RISC-V Cores for AI and Beyond,” at the RISC-V Summit on December 13, 2022.
The RISC-V community – including the technical, industry, domain and special interest groups who define the architecture’s specifications – attend the RISC-V Summit for four days of technology breakthroughs, industry milestones, tutorials, and relationship building. The event runs December 13–14, 2022 in San Jose, CA.
Presentation Title: Real World Results using Thousands of RISC-V Cores for AI and Beyond
Speaker: Dave Ditzel, CTO and Founder, Esperanto Technologies, Inc.
When: Tuesday December 13, 2022 4:45pm - 5:05pm PST
Where: Grand Ballroom A, San Jose McEnery Convention Center, 150 W San Carlos Street, San Jose, CA 95113
Abstract: Esperanto’s ET-SoC-1 features over a thousand 64-bit RISC-V vector/tensor cores on a single 7nm chip. This product started shipping to customers in 2022. This presentation will discuss the architecture of the ET-SoC-1 and how it is incorporated into systems appropriate for data centers. This presentation will present real world results in both performance and performance per watt. We will show a system example where a single rack could hold over 300,000 RISC-V processors. The results demonstrate that one can achieve excellent results for accelerating machine learning and other applications by building on top of the general-purpose RISC-V instruction set. One does not have to resort to GPUs to get good performance per watt, RISC-V is a better alternative. This talk will also discuss Esperanto’s roadmap, which will incorporate chiplet inter-operability and much higher performance, improved performance per watt, and larger number of cores per package.
Track: Industry, High-Performance Computing & Data Centers
About RISC-V International
RISC-V International is the non-profit home of the open standard RISC-V Instruction Set Architecture (ISA), related specifications, and stakeholder community. More than 2,900 RISC-V members across 70 countries contribute and collaborate to define RISC-V open specifications as well as convene and govern related technical, industry, domain, and special interest groups. RISC-V combines a modular technical approach with an open, royalty-free license model — meaning that anyone, anywhere can benefit from the IP contributed and produced by RISC-V. RISC-V enables the community to share technical investment, contribute to the strategic future, create more rapidly, enjoy unprecedented design freedom, and substantially reduce the cost of innovation. To learn more, visit www.riscv.org.
RISC-Vの進化を牽引するEsperantoの挑戦。シンプルなアイデアと実装の困難さ―Dave Ditzelへのインタビュー
Featured article in gihyo.jp by 安田豊（やすだゆたか (Yutaka Yasuda)
RISC-Vの
Read more here in Japanese.....
Read the auto-translated article in English here....
Esperanto Delivers Keynote at MICRO 2022
Hear Dave Ditzel of Esperanto Technologies at MICRO 2022
Dave Ditzel, Founder and CTO of Esperanto Technologies, will deliver a keynote, “The RISC journey from one to a million processors,” at MICRO on Oct 3.
MICRO 2022 is the 55th IEEE/ACM International Symposium on Microarchitecture®.
The event runs October 1–5, 2022 in Chicago, Illinois.
Keynote: “The evolution of processor design in the last 50 years has taken a variety of paths, but the principles embodied in RISC design have been one of the most used approaches over the last 25 years. This talk will make the case that general-purpose RISC processors are likely to stay in the mainstream for the foreseeable future, along with other key technology developments that will influence the design of processors of the future. Having been an active participant in the RISC revolution since the term was coined in 1980, this talk will review some of the salient achievements in the evolution of RISC until today, and talk about where the future will lead us. Most notable is that until recently, much computing has been done on only a single or small number of processor cores, but that we are entering a renaissance where access to thousands to millions of processor cores will be as common as a desktop computer is today. This talk will touch on some of the key technology advances that will enable our RISC based future and make some predictions for what we can expect in the next decade.” – Dave Ditzel
Click the link below to download Dave’s slides:
About IEEE MICRO 2022
The IEEE/ACM International Symposium on Microarchitecture® is the premier forum for presenting, discussing, and debating innovative microarchitecture ideas and techniques for advanced computing and communication systems. For more conference information, https://www.microarch.org/micro55/