The code
For this experiment I followed this Microsoft guidance to use Parallel.ForEachAsync
.
The full code can be seen at 899df9, but the key method looks like this:
private async Task CreateAndDeleteAccounts(int numberOfAccounts)
{
var accountsToCreate = new List<Entity>();
var count = 0;
while (count < numberOfAccounts)
{
var account = new Entity("account")
{
["name"] = $"Account {count}"
};
accountsToCreate.Add(account);
count++;
}
var createdIds = new ConcurrentBag<Guid>();
try
{
_logger.LogInformation($"Creating and deleting {accountsToCreate.Count} accounts");
var startCreate = DateTime.Now;
var parallelOptions = new ParallelOptions()
{
MaxDegreeOfParallelism =
_xrmService.RecommendedDegreesOfParallelism
};
await Parallel.ForEachAsync(
source: accountsToCreate,
parallelOptions: parallelOptions,
async (entity, token) =>
{
createdIds.Add(await _xrmService.CreateAsync(entity, token));
});
var secondsToCreate = (DateTime.Now - startCreate).TotalSeconds;
_logger.LogInformation($"Created {accountsToCreate.Count} accounts in {Math.Round(secondsToCreate)} seconds.");
_logger.LogInformation($"Deleting {createdIds.Count} accounts");
var startDelete = DateTime.Now;
await Parallel.ForEachAsync(
source: createdIds,
parallelOptions: parallelOptions,
async (id, token) =>
{
await _xrmService.DeleteAsync("account", id, token);
});
var secondsToDelete = (DateTime.Now - startDelete).TotalSeconds;
_logger.LogInformation($"Deleted {createdIds.Count} accounts in {Math.Round(secondsToDelete)} seconds.");
}
catch (AggregateException ae)
{
var inner = ae.InnerExceptions.FirstOrDefault();
if (inner != null)
{
throw inner;
}
}
}
The experiment
On the first trial I tried to create/delete 1,000 accounts, on a similar setup to last time, running with MaxDegreeOfParallelism
set to the reported RecommendedDegreesOfParallelism
from the ServiceClient
instance. This performed ~7% faster than the synchronous approach in experiment 1 - a total of 469 seconds of which 37 seconds were for record deletion.
As a small tweak I ran the code again but with doubled MaxDegreeOfParallelism
and this was noticeably worse overall, running into a lot of long retry timeouts because of aggregated processing time: a total of 769 seconds (of which 87 were to delete).
Reflections
- perhaps not as much improvement as I expected
- tuning the
MaxDegreeOfParallelism
is critical - next steps will be to try using
HttpClient
directly, withPolly
to implement the retry logic
See also
#100DaysToOffload 8/100