Mapping with Flow Guide

Inbound mapping is the term to mean the inbound data flow into Salesforce from an external system. The inbound mapping process usually is performed using Batch Apex because the number of records to process can exceed the Salesforce Governor limits. This guide will show you how to configure a Data Synchronization Integration Scenario to run a Salesforce Flow instead of using the Batch Apex.

 

Advantages

Using a Flow for inbound mapping processing has the following advantages.

 

Considerations

 

Prerequisites

  1. An activated Data Synchronization Integration Scenario.
  2. An inbound object mapping configuration saved on the Scenario.

 

Basic Flow Setup

We need to create a Flow to be run as the replacement for Batch Apex processing.

 

Go to Salesforce Setup -> Process Automation -> Flows -> New Flow -> Autolaunched Flow.

  1. Create a resource for the input variable. This is set by the Overcast Object Mapper when the Flow starts.
    1. Resource Type: Variable
    2. Name: inputs (Must match case exactly)
    3. Data Type: Apex-Defined. Allow multiple values (collection) is unchecked
    4. Apex Class: overcast__ObjectMappingInboundFlowInputs
    5. Availability Outside the Flow: Available for input is checked
  2. Create a resource for the output variable. This is read by the Overcast Object Mapper when the Flow ends. It is optional. It is used to report back to the mapping process.
    1. Resource Type: Variable
    2. Name: outputs (Must match case exactly)
    3. Data Type: Apex-Defined. Allow multiple values (collection) is unchecked
    4. Apex Class: overcast__ObjectMappingInboundFlowOutputs
    5. Availability Outside the Flow: Available for output is checked
  3. Save the Flow. Give it a name related to the Scenario that will use it, like "MyScenario_MappingOrchestration".

Mapping Orchestration

When the Scenario mapping operation is run, the Flow will be loaded with the run context. 

 

Some standard actions performed in the Flow would be

 

The following sections explain how to complete each action. Each action is independent but complimentary to each other.

Run the inbound mapping action for all records and all mapping configurations

  1. Add an Action to the Flow.
  2. Select the Overcast category then select Run Overcast Inbound Mapping action.
  3. Give it a label and name.
  4. For Object for "Records", Select an Integration Object for the Scenario. The Inputs will appear.
  5. Inputs field should be set to Inputs [Apex-defined] ({!inputs}).
  6. Click the toggle for Records to disable.
  7. Specific mappings to run should be disabled.
  8. Advanced / Manually assign variables can be left unchecked.

 

Read a subset of the records from the Integration Object for mapping

  1. Add Get Records to the Flow. 
    1. Specify a Label and Name.
    2. Select the Integration Object to read from.
    3. Set the first filter condition. This will ensure that only the current Scenario run's records are queried.
      • Name equals inputs > inboundRecordsName
    4. Set any additional filters, if required.
    5. Set All records stored.
    6. Set Automatically store all fields.
    7. Save.
  2. Pre-process the records in a loop, if required. 
  3. Add Run Overcast Inbound Mapping action and set the Get Records value to the action's Records field.

Run the mappings with a subset of the mapping configurations

Specify in the Run Overcast Inbound Mapping the names of the mapping configurations. Separate them with a comma; a space is not required.

Exit Flow and Return to Batch Processing

Flow processing is limited to the Salesforce Governor limits so care should be used in ensuring not too many records are returned from the backend system, or manually queried from the Integration Objects.

 

We have a pattern that can be utilized to address one of the considerations stated at the top of this page. The advantage of orchestration through Flow is that it gives you greater control over mapping. In the case when the response data is small, Flow orchestration can significantly reduce the time taken from starting a Data Sync Scenario to when the data is saved/mapped to the target records because it is not run on the Salesforce Batch Apex queue. In this case, the response data must never be large to avoid hitting the governor limit of 10,000 records.

 

To work around this hard requirement, when response data is larger than the governor limit we can use the following pattern.

  1. Query for the count of records in the response integration object.
  2. Use a Decision and create two branches
    1. Default (when record count is under the limit)
    2. Over limit (when record count is over the limit)
  3. The default branch performs the current behavior.
  4. In the Over limit branch path, add an Apex Action
    1. Start Mapping Batch Process
    2. Input parameters
      1. Mapping Run Record ID:  {!inputs.mappingRunRecordId}
  5. End the process after Start Mapping Batch Process.

Now, the flow will handle when too much data is returned by running the Batch Apex mapping processor.

Real-time Scenarios (or HTTP Requests) in Mappings

When needing to make HTTP requests in an object mapping enhancement point, like running real-time scenarios to gather extra data to complete a mapping, you may need to carefully construct your flow to avoid the all-too-common "You have uncommitted work pending" error.

 

You will be able to perform one mapping operation with HTTP requests (below) without special handling. The first mapping Apex action makes the HTTP requests and performs the mappings (copying records). But subsequent HTTP requests will fail without special handling.

 

This section will show you how to run multiple mappings that each run real-time scenarios in the enhancement point Apex code.

 

Follow these steps to create the flow above.

  1. Create a resource for updating the mapping run record.
    1. Resource Type: Variable
    2. Name: mappingRunRecord
    3. Data Type: Record
    4. Object: overcast__Run__c
  2. Add Apex Action element to run the mappings for the first source object.
    1. Type: Run Overcast Inbound Mapping
    2. Name: Run mapping 1
    3. Object: The mapping's source object
    4. Inputs: {!inputs}
  3. Add a Decision based on the mapping success results.
  4. On the Failed path, add an Assignment to set the mapping run record as complete.
    1. Variable: {!Mapping_Run_Record.Id}
      Operator: Equals
      Value: {!inputs.mappingRunRecordId}
    2. Variable: {!Mapping_Run_Record.overcast__StatusCode__c}
      Operator: Equals
      Value: {!Run_mapping_1.statusCode} (This is the output from the last mapping)
  5. On the Default path, add an Assignment to add a log about the following wait.
    1. Variable: {!outputs.intoMessages}
      Operator: Add
      Value: Waiting for Run mapping 2.
    2. Let's also set the status code so the mapping run record is not set as complete before the second mapping is completed.
      Variable: outputs.statusCode
      Operator: Equals
      Value: 110 (This means mapping is still in process)
  6. Add a Wait for Amount of Time element
    1. Label: Transaction Separator
    2. Amount of Time: 1 Minute
  7. Add an Apex Action like 2. to run the second mapping operation.
  8. Add an Assignment to set the mapping run record as complete.
    1. Variable: {!Mapping_Run_Record.Id}
      Operator: Equals
      Value: {!inputs.mappingRunRecordId}
    2. Variable: {!Mapping_Run_Record.overcast__StatusCode__c}
      Operator: Equals
      Value: {!Run_mapping_2.statusCode} (This is the output from the last mapping)
  9. Add an Update Records element to save the mapping run record changes.
    1. Label: Set as finished
    2. How to Find Records to Update and Set Their Values: Use the IDs...
    3. Record: {!Mapping_Run_Record}
  10. End

Handle errors

It is a Salesforce Flow best practice to add a Fault Path to Actions so that users (and admins) do not get ugly error emails. Additionally, when a Fault occurs, the transaction is rolled back and saves are lost. You can create a Fault Path by connecting an action to a second target.

Overcast actions will throw exceptions when there is something that is cannot recover from, like the configuration is bad.

 

Records failing to be mapped by the object mapping operation do not cause a Fault. The error is logged on the mapping run, the mapping run ends with an appropriate error status code, and the Flow action completes normally with the output values Success and Status Code reflecting the status code.        

 

This Flow shows when the mapping operation Map products completes successfully a record is updated with the result.   

 

The Decision element checks the output value Map products > Success equals true. 

 

 

Scenario Configuration

  1. Open the Scenario and edit it.
  2. In the Inbound Mapping Orchestration field, select the Flow you created in Basic Flow Setup.
  3. Set IO Records Deletion to After mapping each batch.
    This is set so records can be deleted in the same transaction. In the case of too many records for a transaction, setting this allows the Scenario mapping operation to be rerun to process the remaining records. If After mapping each batch is not set and too many records are required to be mapped, then the mapping operation is terminated without performing any mapping.
  4. Save. There is no need to Activate.

 

Configuration complete. Now you can run the Scenario and your flow will run to process the records.