Google is set to launch a new data lake storage engine dubbed BigLake.

The BigTech said the new product will allow enterprises to break down the barriers between their data warehouses and data lakes, providing a unified interface across storage layers, according to sources reported by ZDNet.

A data lake is a system or repository of data stored in its natural or raw format, usually as “object blobs” or files.

The sources said that BigLake will be able to support open file formats such as Parquet, as well as open source-processing engines like Apache Spark or Beam, and table formats such as Delta and Iceberg.

BigLake is set to become a central part of all Google Cloud’s investments going forward according to the sources.

Additionally, Google announced the formation of the Data Cloud Alliance, in partnership with firms such as Confluent, Databricks, Dataiku, Deloitte, Elastic, Fivetran, MongoDB, Neo4j, Redis, and Starburst.

The BigTech said the new alliance will ensure that global businesses have more access and insights into the data required for digital transformation.

The core principles laid out by the new group include “accelerating adoption across industries through common industry data models, open standards, processes, and end-to-end integrated products and solutions”, as well as “reducing challenges and complexity with data governance, data privacy, data loss prevention, and global compliance”, and “solving the skills gap through skill development for practitioners in modern data and analytics technologies”.

"We are excited to partner with Google Cloud and the members of the Data Cloud Alliance to unify access to data across clouds and application environments,” said Mark Porter, chief technology officer at MongoDB. “Legacy frameworks have made working with data hard for too many organisations.”

He added: “There couldn’t be a more timely and important data initiative to build faster and smarter data-driven applications for customers."

Share Story: