Dataiku Embraces Containers to Advance AI

Dataiku, a provider of a framework through which developers of artificial intelligence (AI) application collaborate, this week announced that its software now can create a Docker image containing AI code and required libraries or packages and automatically deploy it on a Kubernetes cluster.

Dr. Kenneth Sanford, analytics architect lead in the United States for Dataiku, says version 5.0 of the company’s namesake software is designed to be able to more easily distribute in-memory processing of recipes written in Python and R to advance AI model training and scoring.

Organizations are launching AI projects both in the cloud and in on-premises environments. Therefore, support for Docker containers makes it easier to set a framework through which data scientists and line of business executives can collaborate during the project.

Sanford says one of the biggest challenges with AI is that not everyone in the organization understands how the models are created. The lack of visibility into the process tends to create a level of skepticism concerning the validity of the model, he notes, but via the Dataiku Discussions tool, anyone participating in an AI project can create, subscribe or comment on a topic or project. Teams also now can create wiki spaces within Dataiku for collaborative creation and editing of documentation.

Builders of AI models soon will be required to document how they were built to satisfy auditors who are not likely to have a deep background in AI, adds Sandford. Dataiku provides a means for surfacing the documentation in a way most auditors will be able to follow, says Sanford.

Dataiku’s Tensorboard dashboard provides a means to not only provide more transparency into the AI process, but also makes it easier for data scientists to make the case for investing in a project in the first place, says Sanford.

The latest version of Dataiku also adds support for Keras, a neural network library written in Python that is employed within a set of deep learning algorithms. That library complements a wide range of open source machine and deep learning algorithms that Dataiku has helped curate on behalf of more than 200 organizations that already employ the platform.

As in in the case of building traditional applications using a standard set of DevOps processes, the need for a similar approach to AI is becoming apparent. AI models are not built by developers. However, they are based on a set of algorithms that data scientists train to recognize patterns. But over time, organizations will find they need to replace one set of models for another as the data sources being included with the model continue to expand and evolve. Dataiku essentially provides the framework for managing that process.

It’s only a matter of time before AI models are injected into every conceivable enterprise application. End users at a minimum will come to expect the same level of experience in their business applications that they encounter in consumer applications. The only issue left now is deciding how best to deliver on those expectations.

Mike Vizard

Mike Vizard is a seasoned IT journalist with over 25 years of experience. He also contributed to IT Business Edge, Channel Insider, Baseline and a variety of other IT titles. Previously, Vizard was the editorial director for Ziff-Davis Enterprise as well as Editor-in-Chief for CRN and InfoWorld.

Mike Vizard has 1540 posts and counting. See all posts by Mike Vizard