Automate Data Ingestion
at Scale

BDM Ingest automates the ingestion of data at scale while simplifying the creation and operation of your pipelines.

Let Bluemetrix Data Manager simplify your Data Engineering

Most enterprises have multiple different data sources that are created each day. New tables, new files with names you've never heard of - it's a tough job to keep up!

BDM Ingests source connectors and automation functionality have been developed for your unique needs - which means you can rest easy knowing your pipelines are always up to date and ingesting data even as your data sources change.

From the first login to fully fleshed out data pipelines, source to destination, BDM Ingest allows you create pipelines in minutes to manage all of your data ingestion needs.

Product Overview

Completely Automated

BDM Ingest uses automation to create an intelligent ingestion engine that can simplify the ingestion of any number of databases, files or other data sources at scale while maintaining high throughputs and low latencies.

Scalability with Incomparable Speed

BDM uses the native Kubernetes as a Service solution available through your cloud provider, to allow multiple instances of Spark VM’s to be spun up at scale allowing any amount of data from multiple sources to be ingested.

Enterprise Level of Ingestion

BDM runs Spark on Kubernetes, allowing it to be deployed on your cloud of choice (AWS, Azure, GCP) using their native Kubernetes as a Service offering. BDM Ingest also integrates with all major on-premise data sources and environments to enable the integration of cloud and on-premise processing environments.

How BDM Ingest Works

Massive Time Savings and Productivity Gains

With BDM Ingest, your data engineering teams will be able to migrate their data to the cloud with an intuitive solution. Not only does it have the most complete suite of connectors, automation and orchestration features, but the software is free!

Completely Automated

BDM Ingest allows you to automate the ingestion of complex data sources, ensuring that as your data source changes dynamically, your ingestion pipelines remain stable. It works with all major enterprise data sources, ensuring all data can be simply ingested onto the cloud.

Templates: The ability to build and use templates to ingest complex data sources
Variables: The ability to use variables in templates to ingest complex data sources
Large Scale Ingest: We have custom solutions to work with all file data sources
Orchestration: We support multiple scheduler tools to automate the execution of the ingestion
Pipelines: BDM Ingest automates the creation and the management of your pipelines

Book a tour of Bluemetrix

Cloud Environment

Supports all major cloud environments

BDM Ingest use cloud native services on your account

(Kubernetes, AD/ LDAP) to move data from sources to destinations.

Your Kubernetes clusters can run on on-premises or in data lakes and warehouses like Azure, Google Cloud, AWS etc., allowing you to query anything at any time, affordably and securely.

At all stages of the ingestion process, your data will remain in your environment and will only be accessible to your data engineering team.

FAQ

Frequently Asked Questions

Select from the following list of Product and Technical FAQs. Browse through these FAQs to find answers to commonly raised questions about creating Pipelines, configuring Sources and Destinations, and working with Models.

How do I learn more about BDM Ingest?

Reach out here to learn more with BDM Ingest for free and automate data pipeline management with a visual low code builder. Alternatively, you can request a personalized demo from our team.
What data sources does BDM Ingest offer connectors for?

Bluemetrix offers a full suite of Connection Profiles for major data sources - Mainframes, Data Warehouse, Files, Streaming Data - and destinations that includes, Databases: JDBC, etc. Files: JSON, CSV, AVRO, EBCDIC, Text, Parquet, ORC, etc Streams: Kafka & Spark Structured Streaming We also add new connectors based on customer requests. The more requests we get for a source, the higher we prioritize building the new connectors.
How does BDM Ingest automate the ingestion of data?

Bluemetrix has been working with Hadoop and other Data Lake technologies since 2009, and in that time we have built over 400 enterprise Data Lakes. Using this experience we have developed our own proprietary technology to create an intelligent ingestion engine that simplifies the ingestion of data at scale. The functionality includes: Templates: The ability to build and use templates to ingest complex data sources Variables: The ability to use variables in templates to ingest complex data sources Large Scale Ingest: We have custom solutions to work with most data sources Orchestration: We support multiple scheduler tools to automate the execution of the ingestion Pipelines: BDM Ingest automates the creation and the management of your pipelines
How does Bluemetrix handle changes in the source, such as schema or API changes?"

Our pipelines are configured to handle new fields or tables added to your source automatically, so you don’t need to make manual adjustments in the UI. As the schema of your data changes at source, we implement these changes at the destination plus we inform all pipeline owners that consume the source of the changes as they happen, so that they can change their pipelines if necessary. We constantly monitor and stay ahead of API changes or deprecations so you don’t need to think about it.
Do I have to do anything if an API endpoint is changed?

No, the Bluemetrix team will update the connector. BDM Ingest is fully managed, including managing your destination schema in addition to staying ahead of API changes for all connectors.

How do I learn more about BDM Ingest?

Reach out here to learn more with BDM Ingest for free and automate data pipeline management with a visual low code builder. Alternatively, you can request a personalized demo from our team.
What data sources does BDM Ingest offer connectors for?

Bluemetrix offers a full suite of Connection Profiles for major data sources - Mainframes, Data Warehouse, Files, Streaming Data - and destinations that includes, Databases: JDBC, etc. Files: JSON, CSV, AVRO, EBCDIC, Text, Parquet, ORC, etc Streams: Kafka & Spark Structured Streaming We also add new connectors based on customer requests. The more requests we get for a source, the higher we prioritize building the new connectors.
How does BDM Ingest automate the ingestion of data?

Bluemetrix has been working with Hadoop and other Data Lake technologies since 2009, and in that time we have built over 400 enterprise Data Lakes. Using this experience we have developed our own proprietary technology to create an intelligent ingestion engine that simplifies the ingestion of data at scale. The functionality includes: Templates: The ability to build and use templates to ingest complex data sources Variables: The ability to use variables in templates to ingest complex data sources Large Scale Ingest: We have custom solutions to work with most data sources Orchestration: We support multiple scheduler tools to automate the execution of the ingestion Pipelines: BDM Ingest automates the creation and the management of your pipelines
How does Bluemetrix handle changes in the source, such as schema or API changes?"

Our pipelines are configured to handle new fields or tables added to your source automatically, so you don’t need to make manual adjustments in the UI. As the schema of your data changes at source, we implement these changes at the destination plus we inform all pipeline owners that consume the source of the changes as they happen, so that they can change their pipelines if necessary. We constantly monitor and stay ahead of API changes or deprecations so you don’t need to think about it.
Do I have to do anything if an API endpoint is changed?

No, the Bluemetrix team will update the connector. BDM Ingest is fully managed, including managing your destination schema in addition to staying ahead of API changes for all connectors.