Databricks and NVIDIA Expand Partnership to Boost AI Efficiency
Databricks and NVIDIA have announced an expanded partnership to optimise data and AI workloads by integrating NVIDIA CUDA accelerated computing into Databricks’ Data Intelligence Platform. This collaboration promises to significantly enhance AI development pipelines’ efficiency, accuracy, and performance, benefiting modern AI factories. The enhanced partnership was revealed during the Data + AI Summit, emphasising the critical role of data preparation, curation, and processing in utilising enterprise data for generative AI applications.
During #DataAISumit, we announced an expanded collaboration with @NVIDIA. Together, we will optimize data and AI workloads by bringing NVIDIA CUDA-accelerated computing to the core of Databricks’ Data Intelligence Platform. https://t.co/XhLol7PMGbpic.twitter.com/Z3CbjUEf4E
Enhanced Efficiency and Performance with CUDA Acceleration
Ali Ghodsi, Co-founder and CEO of Databricks, expressed enthusiasm about the partnership, stating, “We’re thrilled to continue growing our partnership with NVIDIA to deliver on the promise of data intelligence for our customers from analytics use cases to AI.” Jensen Huang, Founder and CEO of NVIDIA, highlighted the importance of data in the generative AI revolution, noting, “Data is the fuel for the generative AI industrial revolution, so reducing data processing energy demands with accelerated computing is essential to sustainable AI platforms.”
Jensen Huang, Founder and CEO of @nvidia joined Databricks Co-founder and CEO @alighodsi for a fireside chat on innovation in the age of AI.
"Whatever you do, just start," said Huang. "This is a fast-moving train. You don’t want to wait and observe an exponential trend." pic.twitter.com/WMVNXbVEwq
A key aspect of this collaboration is the integration of NVIDIA-accelerated computing into Databricks’ next-generation vectorised query engine, Photon. This integration aims to deliver improved speed and efficiency for data warehousing and analytics workloads. Photon, which powers Databricks SQL, is known for its industry-leading price performance and total cost of ownership (TCO). The collaboration between Databricks and NVIDIA is expected to set new standards in price performance.
Creating Generative AI Factories with NVIDIA NIM
Databricks’ open-source model DBRX has been made available as an NVIDIA NIM microservice, providing fully optimised, pre-built containers for deployment. This development significantly increases enterprise developer productivity by offering a simple, standardised way to incorporate generative AI models into applications. DBRX, built entirely on Databricks and trained with NVIDIA DGX Cloud, allows organisations to customise models with enterprise data to create high-quality, organisation-specific models.
“Every company’s proprietary data is a crucial asset for creating intelligence in the era of AI,” said Jensen Huang, founder and CEO of @nvidia at the #dataaisummit in San Francisco today pic.twitter.com/mZg8ZfsiXX
Comprehensive Platform for Generative AI Applications
The Databricks Data Intelligence Platform is recognised for offering a comprehensive solution for building, evaluating, deploying, securing, and monitoring end-to-end generative AI applications. With Databricks Mosaic AI’s data-centric approach, customers can quickly scale generative AI applications on their unique data, ensuring safety, accuracy, and governance.
Continued Growth and Strategic Acquisitions
This announcement follows Databricks’ strategic acquisition of Tabular, a data management startup, enhancing its capabilities with the creators of Apache Iceberg and Linux Foundation Delta Lake. This acquisition, along with recent product innovations and strategic partnerships, positions Databricks as a leader in breaking down data silos and unlocking AI innovation. Databricks’ recent financial success, with over $1.6 billion in revenue and more than 50% year-over-year growth, underscores the increasing demand for data and AI capabilities.
NVIDIA’s Commitment to AI Innovation
NVIDIA’s commitment to AI innovation is reflected in its impressive financial performance, with a revenue increase of 262% from the same quarter a year earlier. Jensen Huang attributed this growth to businesses and governments partnering with NVIDIA to transition from traditional data centres to AI factories. NVIDIA plans to update its AI accelerators annually, with significant advancements expected in the coming years.
Conclusion
The expanded partnership between Databricks and NVIDIA marks a significant step forward in the evolution of AI and data processing. By integrating NVIDIA’s advanced computing capabilities into Databricks’ robust data platform, the two companies are set to enhance enterprise AI applications’ efficiency, accuracy, and performance. This collaboration promises to drive innovation and pave the way for sustainable and scalable AI solutions across various industries.
