Snowflake Embraces Containers Alongside DevOps Workflows
Snowflake made available today a private preview of an instance of its framework for building and deploying applications that can be deployed as a set of integrated containers. IT teams can also bring their own containers to the Snowflake platform.
Snowpark Container Services, announced at the Snowflake Summit 2023 conference, will make it simpler for application developers to use multiple cloud services to build applications as they best see fit.
At the same time, Snowflake is making available a private preview of support for native Git integration with continuous integration/continuous delivery (CI/CD) platforms and a native command line interface (CLI) that makes the Snowflake platform more accessible to developers.
Snowflake is also previewing a set of application programming interfaces (APIs) and a registry that will make it easier to incorporate services for building artificial intelligence (AI) models from, for example, Microsoft and NVIDIA, into their applications. An existing Streamlit streaming tool will also provide the ability to turn models into interactive maps by adding a few lines of code.
In addition, Snowflake is moving to eliminate boundaries between batch and streaming pipelines via a Snowpipe Streaming framework that will be generally available soon and a preview of Dynamic Tables, a tool that enables data engineers to ingest data in a way that makes it simpler to build more complex pipelines.
Snowpark Container Services is also being incorporated into Snowflake Native App, a framework for distributing applications via an online marketplace that Snowflake is previewing on the Amazon Web Services (AWS) cloud. Snowflake is also committing to making it easier for third-party partners to build their own connectors to its platform.
Torsten Grabs, senior director for product management for Snowflake, said containers will make it much simpler for IT teams to bring processing capabilities to where data already resides rather than moving data into a centralized repository. In the absence of that capability, IT teams will find themselves managing duplicate editions of platforms that complicate everything from data governance to cybersecurity, he added.
Snowflake is making a case for building and deploying applications on top of a platform that also automates the management of the pipelines used by applications to access data. It’s not clear to what degree the Snowflake platform will be employed to host applications, but the company is clearly betting the volume of data residing in its cloud platform will attract a wide range of application developers that, in turn, will be supported by DevOps teams managing workflows on the Snowflake platform.
Of course, Snowflake is not the only provider of a data warehouse or data lake with similar ambitions. Regardless of platform, however, a convergence of DevOps and DataOps workflows is now underway as IT organizations look to streamline the management of the massive amounts of data that now reside in the cloud.
The challenge, as always, is building and deploying applications that access the massive amounts of data at scale.
