Why Data Archival is important in salesforce?
Salesforce works better with Operational data and transaction data but when it comes large data volumes the performance goes down for the Reports, List Views, query performance, while Re assigning the owner or updating the role hierarchy will impact user experience.
So it is always better understand your storage limits and data grown trends well in advance.
How can we avoid it?
- Define your Operational data set and store only that data
- Delete the unnecessary data (Back up before mass deleting data and keep in mind that data integrity implications)
- Delete unnecessary object and their related data.
- Create an archiving policy
Different Archival/Data Backup processes which support in salesforce:
- Shadow Objects: Shadow Object is a custom object which will holds the same structure of base Object. That means same fields, same CRUD, OWD, field level security. We can move the records to this new object based on criteria. In this way we can reduce the volume in base object.
- Heroku : Cloud Based Service to move data to and from Salesforce to Heroku(Postgres). Using bi-directional synchronization between Salesforce and HerokuPostgres, HerokuConnect unifies the data in your Postgres database with the contacts, accounts and other custom objects in the Salesforce database
- Different app exchange Products: The most popular appexchange products to bachup are Backupify, Ownbackup for Salesforce, Spanning Backup, Odaseva
- External Objects: are similar to custom objects in salesforce, But external object record data is stored outside your salesforce Organization. Each external object is associated with an external data source definition in your Salesforce organization. An external data source specifies how to access an external system.
- Big Objects: By using Big objects you can store massive amount of data in salesforce platform. Big objects provide consistent performance for a Billion records or more. There are may more options to do data archival and data backup in salesforce based on requirement. Now Let us discuss more about Big Objects
We have two types of big objects
- Standard big Objects: These are defined by salesforce and included in salesforce products. Ex : FieldHistoryArchive (Which allows you to store upto 10 years of archived field history data)
- Custom Big Objects: You can not create big objects through Standard Slaesforce UI. However you can create big objects by following two ways.
- By using Metadata API : Custom Big Objects are defined and deployed by you through metadata api. To define a custom big object, you create an object file that contains its definition, fields, and index, along with a permissionset to define the permissions for each field, and a package file to define the contents of the object metadata. After inserting metadata file through workbench you can check the Big Object API Name ends with “__b” in org.
There is a trailhead which will explain to create a big object using metadata api.
- Custom Big Object Creator: There is another way to create Big Object is, by installing “Custom Big Object Creator” managed package from Salesforce Labs. We can create Big object using Custom Big object creator lightning tab. In Big Objects OWD, CRUD, Field level security can be set as per requirement. As these objects are using for storage so it will support only 5 data types which are “Text, Long Text Area, DateTime, Number and Look Up”.
Why I can use Custom Big Objects?
- 360 degree view of the customer
- Auditing and tracking
- Historical Archive
How can I insert records to Big Objects?
There are different ways to create BigObject record, like using a csv file, use APIs like Bulk API or even Async SOQL or database.insertImmediate(record) apex method
Example: If I want to move records from CustomObj__c to CustomObjHistory__b using Apex , need to retrieve the records from VustomObj__c and insert records to CustomObjHistory__b,
CustomObj__c actualrecords = [Select id, canbemerged__c, testfield1__c, testfield2__c from CustonObj__c limit 1];
String merged = actualrecords.canbemerged__c == true ? ‘true’ : ‘False’
CustomObjHistory__b cbh = new CustomObjHistory__b();
cbh.canbemerged__c = merged;
cbh.testfield1__c = actualrecords. testfield1__c;
cbh.testfield2__c = actualrecords. Testfield2__c;
database.insertImmediate(cbh);
Using REST API:
You can insert the data into the object by using rest api post method through work bench:
“/Service/Data/v40.0/sObjects/CustomObjHistory__b”
You can expose the data by using visual force page or lightning component.
Big Objects Storage and capacity?
Eventhough Big Objects supports Millions or hundreds of millions or billions of records but the limit of the Big objects is actually limit to 1 million records without any cost. We can increase this limit by buying the storage space. The cost is approximately $16,800 AUD per year per 50M records (we can’t buy it in any smaller amounts)

How can We Query Big Objects?
If we know that you are querying small amount of records then you can use SOQL
And the another way is “Async SOQL”
AsyncSOQL: To manage millions and millions of records in your custom big objects salesforce introduced AsyncSOQL. But Async SOQL is included only with the licensing of additional big object capacity.
Big Objects Considerations:
- You can create only 100 big objects per org. The limits for big object fields are similar to the limits on custom objects, and depend on your org’s license type.
- Big Objects supports only Object and Field level security
- Big Objects can be used in Einstein analytics but not for report builder and search.
- We cannot track field history in Big Objects
- Big Objects don’t support transactions that include big objects, standard objects, and custom objects.
- You cannot write triggers, flows, processes on big objects.
- You cannot use Salesforce Connect external objects to access big objects in another org.
- The best practice when writing to a big object is to have a retry mechanism. (For example Retry the batch until you get a successful result from the API or Apex method)
- You can delete the data in Big Objects using “deleteImmediate()” apex method.














