Dataverse resilience experiment 2

Still trying to bulk create records using ServiceClient’s built-in retry logic, but using async

A second approach to parallel record creation using ServiceClient but using Parallel.ForEachAsync

The code

For this experiment I followed this Microsoft guidance to use Parallel.ForEachAsync.

The full code can be seen at 899df9, but the key method looks like this:

private async Task CreateAndDeleteAccounts(int numberOfAccounts)
{
    var accountsToCreate = new List<Entity>();
    var count = 0;
    while (count < numberOfAccounts)
    {
        var account = new Entity("account")
        {
            ["name"] = $"Account {count}"
        };
        accountsToCreate.Add(account);
        count++;
    }

    var createdIds = new ConcurrentBag<Guid>();

    try
    {
        _logger.LogInformation($"Creating and deleting {accountsToCreate.Count} accounts");

        var startCreate = DateTime.Now;

        var parallelOptions = new ParallelOptions()
        {
            MaxDegreeOfParallelism =
                _xrmService.RecommendedDegreesOfParallelism
        };

        await Parallel.ForEachAsync(
            source: accountsToCreate,
            parallelOptions: parallelOptions,
            async (entity, token) =>
            {
                createdIds.Add(await _xrmService.CreateAsync(entity, token));
            });

        var secondsToCreate = (DateTime.Now - startCreate).TotalSeconds;

        _logger.LogInformation($"Created {accountsToCreate.Count} accounts in  {Math.Round(secondsToCreate)} seconds.");

        _logger.LogInformation($"Deleting {createdIds.Count} accounts");
        var startDelete = DateTime.Now;

        await Parallel.ForEachAsync(
            source: createdIds,
            parallelOptions: parallelOptions,
            async (id, token) =>
            {
                await _xrmService.DeleteAsync("account", id, token);
            });

        var secondsToDelete = (DateTime.Now - startDelete).TotalSeconds;

        _logger.LogInformation($"Deleted {createdIds.Count} accounts in {Math.Round(secondsToDelete)} seconds.");

    }
    catch (AggregateException ae)
    {
        var inner = ae.InnerExceptions.FirstOrDefault();
        if (inner != null)
        {
            throw inner;
        }
    }
}

The experiment

On the first trial I tried to create/delete 1,000 accounts, on a similar setup to last time, running with MaxDegreeOfParallelism set to the reported RecommendedDegreesOfParallelism from the ServiceClient instance. This performed ~7% faster than the synchronous approach in experiment 1 - a total of 469 seconds of which 37 seconds were for record deletion.

As a small tweak I ran the code again but with doubled MaxDegreeOfParallelism and this was noticeably worse overall, running into a lot of long retry timeouts because of aggregated processing time: a total of 769 seconds (of which 87 were to delete).

Reflections

  • perhaps not as much improvement as I expected
  • tuning the MaxDegreeOfParallelism is critical
  • next steps will be to try using HttpClient directly, with Polly to implement the retry logic

See also

#100DaysToOffload 8/100

Avatar
Proactive application of technology to business

My interests include technology, personal knowledge management, social change

Related

Next
Previous