LINQ with Index

Let’s say you have a function that returns list of entities with logs. One entity may have 0 or more logs.

private IEnumerable<Entity> GetDetails(IEnumerable<int> ids)
{
    IEnumerable<Entity> entities = _entityRepository.GetByIds(ids);
    foreach (var entity in entities)
    {
        IEnumerable<Log> logs = _logRepository.GetByEntityId(entity.Id);
        entity.Logs = logs;
    }
}

This code is likely slow, since you call LogRepository on each entity. In most situation (depends on your data profile), it is faster to get entire records with one call, and filter locally.

private IEnumerable<Entity> GetDetailsFaster(IEnumerable<int> ids)
{
    IEnumerable<Entity> entities = _entityRepository.GetByIds(ids);
    IEnumerable<Log> allLogs = _logRepository.GetByEntityIds(ids);
    foreach (var entity in entities)
    {
        IEnumerable<Log> logs = allLogs.Where(x => x.EntityId == entity.Id).ToList();
        entity.Logs = logs;
    }
}

If you have many entity records, this is still slow, especially on allLogs.Where(x => x.EntityId == entity.Id) area. It is because you are going through entire logs for every entity.

We can make this faster by introducing new LINQ extension method IndexBy(). This is how you would use:

private IEnumerable<Entity> GetDetailsFaster(IEnumerable<int> ids)
{
    IEnumerable<Entity> entities = _entityRepository.GetByIds(ids);
    IEnumerable<Log> allLogs = _logRepository.GetByEntityIds(ids);
    Dictionary<int, IEnumerable<Log>> indexedLogs = allLogs.IndexBy(x => x.EntityId);
    
    foreach (var entity in entities)
    {
        if (indexedLogs.ContainsKey(entity.Id))
        {
            IEnumerable<Log> logs = indexedLogs[entity.Id];
            entity.Logs = logs;
        }
    }
}

Source code for IndexBy():

public static class EnumerableExtensions
{
    public static Dictionary<TKey, IEnumerable<TValue>> IndexBy<TKey, TValue>(
        this IEnumerable<TValue> list,
        Func<TValue, TKey> keyFilter)
    {
        var result = list.GroupBy(keyFilter).Where(x => x.Any());

        return result.ToDictionary(x => x.Key, x => x.Select(y => y));
    }
}

You can use IndexBy() for any combination of properties.

Written on May 29, 2015