Talend Extends Docker Container Support
Talend, as part of a summer release of Talend Data Fabric, is now making it possible to containerize data services and routes directly from within the Talend Studio toolset.
Nick Piette, director of product marketing for application programing interface (API) services at Talend, notes that as containers become pervasive within highly distributed enterprise IT environments, the next big challenge is to connect all the containers and their associated APIs at scale to various data sources. The Talend Data Fabric provides the data integration framework that enables that goal and Talend Studio allows those integrations to be deployed as containers within the context of a larger microservices-based application, says Piette.
That effort also makes it possible to apply governance policies to data, such as what data can be accessed by any microservice—a compliance issue that increasingly builders of microservices-based applications are being required to address, adds Piette.
In addition to extending existing support for Docker containers, the summer edition of Talend Data Fabric adds a plug-in that makes it easier to integrate the Talend integration framework within a continuous integration framework without having to rely on any custom proprietary scripts, adds Piette. That capability will go a long way to fostering the inevitable convergence of DevOps and DataOps processes within an IT environment because both teams now can share a common set of Apache Maven or CloudBees-based processes, for example, he says.
Other new capabilities include support for Microsoft Azure and Databricks cloud services within Talend’s Pipeline Designer tool for graphically designing DataOps pipelines and a pay-as-you-go option for Talend Pipeline Designer. Talend also now includes extended native support for Delta Lake, the open source storage layer that optimizes analytics application performance.
Additionally, Talend is now making it possible to manage hybrid instances of Talend Data Fabric via a control plane hosted in the cloud, in addition to embedding machine learning algorithms that recommend ways to optimize data workflows.
Finally, Talend added support for reversible Format Preserving Encryption and in-product real-time chat support.
As IT environments become more complex in the age of containers, Piette says data integration frameworks will need to become as distributed as the application environment they are intended to support. It’s not going to be feasible from a performance perspective to route every request for data through, for example, a cloud service; organizations today require a more flexible, frictionless approach to data integration, he says.
The degree to which DevOps and DataOps processes will converge will vary widely in most organizations, but the rise of microservices based on containers will soon force the issue. On the plus side, the availability of data integration frameworks optimized for containers means the technical impediments for achieving that conversion finally are starting to fall by the wayside. The bigger issue now is the degree to which organizations will be able to overcome the divide between DevOps and DataOps cultures that has existed for decades.
