An organization using Salesforce typically has loads of customer data that need to be uploaded and updated from time to time. So, Salesforce has enabled the use of Bulk API to help you manage huge quantities of data effectively.
This post will define exactly what Bulk API is, when it is used, and how it specifically helps you in Salesforce data management efforts.
Bulk API Defined
Bulk API is a functionality grounded on the REST principles and is used to process large amounts of data in batches. This is a programmatic option that can be used by someone who has a background in web services and software development.
Bulk API can process records loaded as CSV files within Salesforce Data Loader. It can also process millions of records through writing your own application, depending on your client.
When Is Bulk API Used?
Bulk API can be used for any amount of records that need processing. Salesforce actually uses the SOAP API to upload and update fewer records while real-time client applications are running. But if you need to upload more than 10,000 records (even up to millions of records), it is a faster and more practical to use Bulk API instead.
You can submit batches of records with a maximum limit of 10,000 records per batch. The maximum limit for batches to be submitted and processed per day is 2,000 batches. Note that submitting such a large number of data means that Salesforce will process them on a rolling basis, and not necessarily simultaneously.
How Does Bulk API Work?
Bulk API can be used to asynchronously insert, upsert, update, delete, query, or queryAll a large volume of records. It specifically works in the following manner:
- Start by creating a new job. Objects to be processed and the action to be performed must be clearly specified here.
- Group data into batches and send all of them to the server via an HTTP POST request.
- Close the job once done sending data batches. The job cannot be reopened once closed.
- You may now check batch status through a BatchInfo resource.
- Retrieve batch results once all of their statuses are done.
- Check against the original data sets to see which data batches have truly been successfully uploaded and which ones failed.
The job can be stopped anytime by using the Abort Job function. Note that all completed data processing will not be affected once you decide to abort the job. Only those who are yet to be processed will be stopped.
Bulk data uploading through this API is made simpler and easier, as Salesforce processes all these data batches in the background while you work on other things.
Summary
Salesforce lets users utilize the Bulk API for processing large quantities of data. The API can manage a few thousands to millions of data asynchronously. It requires knowledge and experience in web development and programming to correctly use it.